queue management algorithms

[ik]

On Sun, Apr 7, 2013 at 6:36 PM, Orna Agmon Ben-Yehuda <ladypine at gmail.com>wrote:

> If you are looking for something reliable for many tasks scheduled at a
> time (e.g., one user may send a hundred tasks within one second) go for
> slurm. openPBS, torque, condor cannot take the heat.  I heard that PBS (not
> open) can, too.
>

One of the issues, is knowing what data to take,  and there is a need for a
cycle. each cycle have finite number of data inside the queue, but each new
cycle recreate the queue to the actual content to be used.

I still research this one.


>
>
> On Sun, Apr 7, 2013 at 5:30 PM, ik <idokan at gmail.com> wrote:
>
>> On Sun, Apr 7, 2013 at 3:51 PM, Oleg Goldshmidt <pub at goldshmidt.org>wrote:
>>
>>>
>>>
>>> On Sun, Apr 7, 2013 at 2:27 PM, ik <idokan at gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I have a very complex queue management requirements, I need to
>>>> calculate task weight, time restrictions for execution, number of possible
>>>> execution at given time, changes of the data when it is inside the queue
>>>> and even disabling specific task.
>>>> some tasks have very short time to live, and required to be executed
>>>> faster, then others. but it's not a "simple" priority queue.
>>>>
>>>
>>> It is difficult to say from the description, but it sounds to me like
>>> your problem is that of "scheduling" rather than just "queue management".
>>> Note that a "scheduler" (e.g., that of the kernel) will have many queues
>>> holding tasks of different types and properties, and will need to manage
>>> all of them together.
>>>
>>> In particular, some of the things you mention sound to me like "real
>>> time" requirements. If you look at, say, the Linux scheduler it handles RT
>>> tasks separately from the "normal" tasks. You may need to do the same.
>>>
>>> Another thing that is missing is the target system. A single server? A
>>> cluster or multicomputer? Something else?
>>>
>>> I'd suggest googling for "scheduling" and "resource management" (of
>>> which "scheduling" is a component). Apart from things like SLURM and Condor
>>> that Orna mentioned, maybe Maui? There are all sorts of proprietary
>>> offerings as well.
>>>
>>
>> At first glance, condor looks like the thing that can help me most.
>>
>> Thanks
>>
>>
>>>
>>> --
>>> Oleg Goldshmidt | pub at goldshmidt.org
>>>
>>
>>
>> _______________________________________________
>> Linux-il mailing list
>> Linux-il at cs.huji.ac.il
>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>>
>>
>
>
> --
> Orna Agmon Ben-Yehuda.
> http://ladypine.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20130407/ba5190d8/attachment.html>