Moving away from cron jobs to some workflow manager

[Steve Litt]

On Tue, 19 Jun 2018 09:42:35 +0300
Omer Zak <w1 at zak.co.il> wrote:

> For dependency management, you may want to use 'make' or modern

Hi Omer,

While corresponding with someone offlist, I had another idea maybe as
good as using make. I could make a customized installation of the
process supervisor part of either the runit or s6 inits, or maybe even
use just plain daemontools, to make sure apps don't run until apps upon
which they depend have finished. So, for appC that depends on output
from appA and appB, then the run script for appC would look something
like the following:

#!/bin/sh
if appAnot_finished; then
  sleep 60   # prevent excessive polling
elif appBnot_finished; then
  sleep 60   # prevent excessive polling
else
  exec appC
fi

If every app expresses its immediate prerequisites that way, the whole
thing will run very efficiently, and in many cases, in parallel where
not prevented by unfinished prerequisites.

Some more complexity would need to be added in order that appA and appB
don't start again before the entire bucket brigade finishes.

 
SteveT

Steve Litt 
June 2018 featured book: Twenty Eight Tales of Troubleshooting
http://www.troubleshooters.com/28