<html><head></head><body>Try GNUbatch. <br>
<br><br><div class="gmail_quote">On June 19, 2018 9:42:35 AM GMT+03:00, Omer Zak <w1@zak.co.il> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k9mail">For dependency management, you may want to use 'make' or modern<br>equivalents ('ant', 'gradle', etc.).<br>For controlling remote nodes, 'ansible' may be able to do the work.<br><br>--- Omer Zak<br><br><br>On Tue, 2018-06-19 at 09:06 +0300, Rabin Yasharzadehe wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"> Hi all, <br> <br> I need some advice, currently I have a huge cron file which schedules<br> tasks one after anther, and each task is position precisely (with<br> some room for error) to start after it predecessor.<br> <br> So if one job start at 00:00 and it will go and fetch some files and<br> it takes 3minutes<br> the next job will be after start right after at ~00:05 <br> and so on <br> <br> the problem is that if one job fails, all other jobs which are depend<br> on him will fail as well, and then I get a shitload of alerts, and<br> the worst part is that if i have to manually start a batch process I<br> need to go to each machine and manually start each job in the right<br> order,<br> <br> I was looking to resolve this problem with a tool which can manage<br> this "pipe line" <br> and I cam across several tools like Luigi and (apache-)AirFlow, I<br> started with Luigi but It didn't look<br> right for the job, and then I tried airflow, but was not able to make<br> it to work, the jobs queue never executed. =(<br> <br> Has any one have experience with airflow, or other tool like it which<br> they can recommend ? <br> My needs are to be able to execute my CURRENT shell/python/php<br> scripts and build the dependency between them, and I perfer the<br> option for remote exec so that I will have central <br> place to manage and monitor all work flow whichs are executed on<br> several nodes.<br></blockquote>-- <br>More proof the End of the World has started. Just saw this online:<br>I think it's beginning! Ten minutes ago there was a group of people<br>waiting at the bus stop outside my house. Now, they're all gone!<br>My own blog is at <a href="https://tddpirate.zak.co.il/">https://tddpirate.zak.co.il/</a><br><br>My opinions, as expressed in this E-mail message, are mine alone.<br>They do not represent the official policy of any organization with which<br>I may be affiliated in any way.<br>WARNING TO SPAMMERS: at <a href="https://www.zak.co.il/spamwarning.html">https://www.zak.co.il/spamwarning.html</a><br><br><hr><br>Linux-il mailing list<br>Linux-il@cs.huji.ac.il<br><a href="http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il">http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il</a><br></pre></blockquote></div></body></html>