<div dir="ltr"><br><div class="gmail_quote">On Sun, Apr 7, 2013 at 7:21 PM, ik <span dir="ltr"><<a href="mailto:idokan@gmail.com" target="_blank">idokan@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote"><div class="im">On Sun, Apr 7, <a href="tel:2013" value="+9722013" target="_blank">2013</a> at 6:36 PM, Orna Agmon Ben-Yehuda <span dir="ltr"><<a href="mailto:ladypine@gmail.com" target="_blank">ladypine@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">If you are looking for something reliable for many tasks scheduled at a time (e.g., one user may send a hundred tasks within one second) go for slurm. openPBS, torque, condor cannot take the heat. I heard that PBS (not open) can, too.<br>
</div></blockquote><div><br></div></div><div>One of the issues, is knowing what data to take, and there is a need for a cycle. each cycle have finite number of data inside the queue, but each new cycle recreate the queue to the actual content to be used.<br>
<br></div><div>I still research this one.<br></div><div class="im"><div> </div></div></div></div></div></blockquote><div> </div><div><div>With task schedulers, you can cancel jobs that are waiting in the queue easily, even with boinc (which is designed for loose connectivity, that is, it does not assume you can always contact the worker machines, so it fits volunteer computing, like SETI-at-home).</div>
</div><div>However, if what you mean by "recreating the actual content to be used" is that you want to change the data on which the jobs operate, then it is even easier. Just have each job pull its data when it starts running from a known place. Then you can change this known place at no central cost until the job starts running.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
</div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Sun, Apr 7, <a href="tel:2013" value="+9722013" target="_blank">2013</a> at 5:30 PM, ik <span dir="ltr"><<a href="mailto:idokan@gmail.com" target="_blank">idokan@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div>On Sun, Apr 7, <a href="tel:2013" value="+9722013" target="_blank">2013</a> at 3:51 PM, Oleg Goldshmidt <span dir="ltr"><<a href="mailto:pub@goldshmidt.org" target="_blank">pub@goldshmidt.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><div>On Sun, Apr 7, <a href="tel:2013" value="+9722013" target="_blank">2013</a> at 2:27 PM, ik <span dir="ltr"><<a href="mailto:idokan@gmail.com" target="_blank">idokan@gmail.com</a>></span> wrote:<br>
</div><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div><div><div>Hello,<br><br></div>I have a very complex queue management requirements, I need to calculate task weight, time restrictions for execution, number of possible execution at given time, changes of the data when it is inside the queue and even disabling specific task.<br>
</div><div>some tasks have very short time to live, and required to be executed faster, then others. but it's not a "simple" priority queue.<br></div></div></div></div></blockquote></div><div><br>It is difficult to say from the description, but it sounds to me like your problem is that of "scheduling" rather than just "queue management". Note that a "scheduler" (e.g., that of the kernel) will have many queues holding tasks of different types and properties, and will need to manage all of them together. <br>
<br>In particular, some of the things you mention sound to me like "real time" requirements. If you look at, say, the Linux scheduler it handles RT tasks separately from the "normal" tasks. You may need to do the same.<br>
<br>Another thing that is missing is the target system. A single server? A cluster or multicomputer? Something else?<br><br>I'd suggest googling for "scheduling" and "resource management" (of which "scheduling" is a component). Apart from things like SLURM and Condor that Orna mentioned, maybe Maui? There are all sorts of proprietary offerings as well.<span><font color="#888888"><br>
</font></span></div></div></div></blockquote><div><br></div></div></div><div>At first glance, condor looks like the thing that can help me most.<br><br></div><div>Thanks<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_quote"><div><span><font color="#888888">
</font></span></div></div><span><font color="#888888"><br><span><font color="#888888">-- <br>Oleg Goldshmidt | <a href="mailto:pub@goldshmidt.org" target="_blank">pub@goldshmidt.org</a>
</font></span></font></span></div>
</blockquote></div><br></div></div>
<br></div></div><div>_______________________________________________<br>
Linux-il mailing list<br>
<a href="mailto:Linux-il@cs.huji.ac.il" target="_blank">Linux-il@cs.huji.ac.il</a><br>
<a href="http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il" target="_blank">http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il</a><br>
<br></div></blockquote></div><div><br><br clear="all"><br>-- <br>Orna Agmon Ben-Yehuda.<br><a href="http://ladypine.org" target="_blank">http://ladypine.org</a>
</div></div>
</blockquote></div></div><br></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Orna Agmon Ben-Yehuda.<br><a href="http://ladypine.org">http://ladypine.org</a>
</div>