queue management algorithms

[Orna Agmon Ben-Yehuda]

If you are looking for something reliable for many tasks scheduled at a
time (e.g., one user may send a hundred tasks within one second) go for
slurm. openPBS, torque, condor cannot take the heat.  I heard that PBS (not
open) can, too.


On Sun, Apr 7, 2013 at 5:30 PM, ik <idokan at gmail.com> wrote:

> On Sun, Apr 7, 2013 at 3:51 PM, Oleg Goldshmidt <pub at goldshmidt.org>wrote:
>
>>
>>
>> On Sun, Apr 7, 2013 at 2:27 PM, ik <idokan at gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I have a very complex queue management requirements, I need to calculate
>>> task weight, time restrictions for execution, number of possible execution
>>> at given time, changes of the data when it is inside the queue and even
>>> disabling specific task.
>>> some tasks have very short time to live, and required to be executed
>>> faster, then others. but it's not a "simple" priority queue.
>>>
>>
>> It is difficult to say from the description, but it sounds to me like
>> your problem is that of "scheduling" rather than just "queue management".
>> Note that a "scheduler" (e.g., that of the kernel) will have many queues
>> holding tasks of different types and properties, and will need to manage
>> all of them together.
>>
>> In particular, some of the things you mention sound to me like "real
>> time" requirements. If you look at, say, the Linux scheduler it handles RT
>> tasks separately from the "normal" tasks. You may need to do the same.
>>
>> Another thing that is missing is the target system. A single server? A
>> cluster or multicomputer? Something else?
>>
>> I'd suggest googling for "scheduling" and "resource management" (of which
>> "scheduling" is a component). Apart from things like SLURM and Condor that
>> Orna mentioned, maybe Maui? There are all sorts of proprietary offerings as
>> well.
>>
>
> At first glance, condor looks like the thing that can help me most.
>
> Thanks
>
>
>>
>> --
>> Oleg Goldshmidt | pub at goldshmidt.org
>>
>
>
> _______________________________________________
> Linux-il mailing list
> Linux-il at cs.huji.ac.il
> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>
>


-- 
Orna Agmon Ben-Yehuda.
http://ladypine.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20130407/95a5f984/attachment.html>