Thursday, July 23, 2009

Flexible scheduling

As explained in my previous post, I needed a flexible way to schedule a small script to run, with variable parameters.

The basic requirements were:
  • simple to configure (no messing in crontabs)

  • flexible enough to have different schedules on different weekdays (on Sundays a job should be started at another time than on Mondays, for example)

  • should offer the possibility to start different jobs at different times

  • I want a log of all commands executed


For instance, I need the command "foo start 1234" to run every day except Sunday at 23:00h, and "foo stop 1234" always at 01:00h.
I also need "bar load 1 2 3" to run every Wednesday at noon and "bar load 4" every other weekday at noon.

And, as I am a Slackware fan, I want to do all configuration by changing a text file. No graphical interface please, as my servers do not even have X installed.

So the challenge was there :)

Using cron
Now I thought of a solution how these scripts should be started or controlled.
The standard crontab (in /var/spool/cron/crontabs/root) could probably control everything I needed, but it would be a cumbersome task to change it every time. But - on the other hand - I did not want to substitute cron, as it does a nice job scheduling simple daily or monthly tasks.

So I decided to use cron to start my own scheduler. As an interval I choose a five minute period, as that would be accurate enough for me. I don't need any job to start at 12:42 exactly, 12:40 or 12:45 if good enough for me.

My script to check all schedules was named "sched5" and I added the following lines to the crontab:

# Special 5-minute-interval job
0 * * * * /usr/local/sbin/sched5 1> /dev/null
5 * * * * /usr/local/sbin/sched5 1> /dev/null
10 * * * * /usr/local/sbin/sched5 1> /dev/null
15 * * * * /usr/local/sbin/sched5 1> /dev/null
20 * * * * /usr/local/sbin/sched5 1> /dev/null
25 * * * * /usr/local/sbin/sched5 1> /dev/null
30 * * * * /usr/local/sbin/sched5 1> /dev/null
35 * * * * /usr/local/sbin/sched5 1> /dev/null
40 * * * * /usr/local/sbin/sched5 1> /dev/null
45 * * * * /usr/local/sbin/sched5 1> /dev/null
50 * * * * /usr/local/sbin/sched5 1> /dev/null
55 * * * * /usr/local/sbin/sched5 1> /dev/null

As can be seen, the script is stored in /usr/local/sbin, in the path for root - as normal users do not need to access it.

The script
The complete script is copied here:

#!/bin/bash
#
# sched5 Check every 5 minutes /etc/sched5.conf
# for possible jobs to run
# Needs to be started every 5 minutes from cron
# Definition of schedules in /etc/sched5.conf
#
# Version: 0.0.1 - Thursday, Jul 23, 2009
#
# Author: Niels Horn (niels.horn@gmail.com)


###################
## Configuration ##
###################

# Standard interval
SCHED5_INTERVAL=5

# Default log
SCHED5_LOG=/var/log/sched5.log

# Default log level:
# 0 = no logging
# 1 = log execution of jobs
# 2 = log everything
SCHED5_LL=1

# Read configuration from /etc/sched5.conf
. /etc/sched5.conf


####################
## Get DAY / TIME ##
####################
DAY=`date +%u`
TIME=`date +%H:%M`
[ $SCHED5_LL -ge 2 ] && \
   echo "Started: Day=$DAY Time=$TIME" >> $SCHED5_LOG


####################################
## Main loop: check all schedules ##
####################################

for sched in `seq $SCHED_NUM`; do

   sched_name=SCHED${sched}_NAME
   eval sched_name=\$$sched_name

   [ $SCHED5_LL -ge 2 ] && \
      echo "-- Schedule $sched = $sched_name" >> $SCHED5_LOG

   # get rules for this schedule
   sched_rules=SCHED${sched}_RULES
   eval sched_rules=\$$sched_rules

   # loop through all rules for this schedule
   cmd_found=0
   for rule in $sched_rules; do

      [ $SCHED5_LL -ge 2 ] && \
         echo " -- Rule $rule " >> $SCHED5_LOG

      # dissect rule
      rule_day=`echo $rule | cut -f1 -d,`
      rule_tim=`echo $rule | cut -f2 -d,`
      rule_cmd=`echo $rule | cut -f3 -d,`
      # calculate time+interval
      timh=${rule_tim:0:2}
      timm=${rule_tim:3}
      [ "${timm:0:1}" == "0" ] && timm=${timm:1}
      let timm=$timm+$SCHED5_INTERVAL
      [ ${#timm} -eq 1 ] && timm="0"$timm
      tim2=$timh":"$timm

      [ $SCHED5_LL -ge 2 ] && \
         echo " -- Rule: $rule_day $rule_tim-$tim2 -> $rule_cmd" >> $SCHED5_LOG

      # Check if rule is valid for this day
      if [ "$rule_day" == "*" -o "$rule_day" == "$DAY" ] ; then

         [ $SCHED5_LL -ge 2 ] && \
            echo " -- day ok" >> $SCHED5_LOG

         # Current time => time of rule?
         if [ "$TIME" == "$rule_tim" -o "$TIME" \> "$rule_tim" ] ; then

            [ $SCHED5_LL -ge 2 ] && \
               echo " -- time start ok" >> $SCHED5_LOG

            # Current time < time of rule + interval?
            if [ "$TIME" \< "$tim2" ] ; then

               [ $SCHED5_LL -ge 2 ] && \
                  echo " -- time end ok" >> $SCHED5_LOG
               # ok, we found a command to execute!
               cmd_found=$rule_cmd
            fi
         fi
      fi
   done # all rules for this schedule

   # Did we find a command to execute?
   if [ "$cmd_found" != "0" ] ; then

      [ $SCHED5_LL -ge 2 ] && \
         echo " -- Starting command $cmd_found" >> $SCHED5_LOG

      # Get command to execute
      cmd=SCHED${sched}_CMD_$cmd_found
      eval cmd=\$$cmd

      [ $SCHED5_LL -ge 1 ] && \
         echo "$TIME Schedule $sched / $sched_name: Starting command \"${cmd}\"" >> $SCHED5_LOG

      # Execute command
      . $cmd

   fi

done # all schedules

[ $SCHED5_LL -ge 2 ] && \
   echo "Finished" >> $SCHED5_LOG

# all done!

Save this script as /usr/local/sbin/sched5 and chmod it to 700, as only root needs to read and execute it. It can also be downloaded from my site.

The configuration file
The configuration file is saved as /etc/sched5.conf and is a simple text file.
To explain to options, I put the examples I gave in the beginning of this post in this configuration:

#!/bin/bash
### Configuration for 'sched5' ##############################################
##
## requires sched5 to be started every 5 minutes from crontab
##

#########################
## Standard Parameters ##
#########################

### Location of log
# Default = /var/log/sched5.log
#SCHED5_LOG=/var/log/sched5.log

### Logging Level
# 0 = log nothing
# 1 = log execution of jobs
# 2 = log everything
# Default = 1
#SCHED5_LL=2


###############
## Schedules ##
###############

# Number of defined schedules
SCHED_NUM=2

##
## Start of Schedule 1 (SCHED1)
## Repeat following lines for each schedule
## Changing the variables from 'SCHED1' to 'SCHED2', 'SCHED3', etc.
##

# Name of schedule, only used in log
SCHED1_NAME=foo_schedule

# Commands to execute
# valid commands are SCHEDx_CMD_1 to SCHEDx_CMD_9
SCHED1_CMD_1='foo start 1234'
SCHED1_CMD_2='foo stop 1234'

# Rules for execution: day,time,cmd
# where day = day to execute (1=Mo,2=Tu,...7=Su, *=all)
# time = time to execute command
# cmd = # of command to execute (1-9, 0=do nothing)
# Separate rules with space, later rule has preference over earlier rule,
# so you can start with a general rule for all days (*), then alter for
# specific days (like 7=Sunday)
# Use cmd=0 to cancel a more general rule (like day=*)
SCHED1_RULES="\
*,23:00,1 \
7,23:00,0 \
*,01:00,2"

##
## End of Schedule 1 (SCHED1)
##

SCHED2_NAME=bar_schedule
SCHED2_CMD_1='bar load 1 2 3'
SCHED2_CMD_2='bar load 4'
SCHED2_RULES="\
*,12:00,2 \
3,12:00,1"


Dissecting the configuration file
The first lines define the level of logging and where the log should be saved.
The defaults are explained in the text.

In this example we have two different schedules:

# Number of defined schedules
SCHED_NUM=2


The first schedule has more comments, to explain all options.

First we see the name of the schedule, only used in the log:

# Name of schedule, only used in log
SCHED1_NAME=foo_schedule


Then we'll find the several commands we can schedule:

# Commands to execute
# valid commands are SCHEDx_CMD_1 to SCHEDx_CMD_9
SCHED1_CMD_1='foo start 1234'
SCHED1_CMD_2='foo stop 1234'

All commands must be executable from the command line by root to function.
We can actually have several different commands, and to be exact, names can also end in words, like "SCHED1_CMD_start" - just change the code from "1" to "start" in the rules below.

We now go to the rules of this schedule:

# Rules for execution: day,time,cmd
# where day = day to execute (1=Mo,2=Tu,...7=Su, *=all)
# time = time to execute command
# cmd = # of command to execute (1-9, 0=do nothing)
# Separate rules with space, later rule has preference over earlier rule,
# so you can start with a general rule for all days (*), then alter for
# specific days (like 7=Sunday)
# Use cmd=0 to cancel a more general rule (like day=*)
SCHED1_RULES="\
*,23:00,1 \
7,23:00,0 \
*,01:00,2"

As you can see, the special command "0" cancels a previous rule. So in this example, we always run command 1 (foo start 1234) at 23:00h, except on Sundays (day=7).
The second rule is simpler: start command 2 every day at 01:00h.

The second schedule has no comments, to show how simple it is to add extra schedules:

SCHED2_NAME=bar_schedule
SCHED2_CMD_1='bar load 1 2 3'
SCHED2_CMD_2='bar load 4'
SCHED2_RULES="\
*,12:00,2 \
3,12:00,1"


Starting it all
After changing the crontab, saving the script and setting up the configuration, make sure that everything is scheduled by restarting crond:

killall crond
/usr/sbin/crond -l10 >>/var/log/cron 2>&1

This last line comes from /etc/rc.d/rc.M

As the configuration file is re-read every time the script starts, you can just edit the /etc/sched5.conf file and the changes will be in effect immediately.


As always, feel free to comment on this post with questions or suggestions!

Labels: , , ,

2 Comments:

Anonymous gmbastos said...

Greetings, Niels.

I just think cron isn't that bad but, sure, a front-end would be highly appreciated by many people.

I'm dropping in this comment just to tell you that "crontab -e" is a better way to edit your crontab: the command is shorter and it handles crontab restarting for you.

Another "crontip": instead of adding 12 entries in order to run the same command each five minutes, you could simply use a "*/5 * * * *" instead.

As an example, experiment adding the following line to your crontab:

*/2 * * * * date "cron command run on: +%Y%m%d-%H:%M" >> ~/cronlog.log

Then you go take a coffe, return and issue a "crontab -e" and comment out that line, otherwise it would fill your hard drive with test-junk. Finally, a "cat ~/cronlog.log" will show the results.

So your crontab could look like this, instead:

# Special 5-minute-interval job
*/5 * * * * /usr/local/sbin/sched5 1> /dev/null

Have fun. :)

October 12, 2009 at 4:55 AM  
Blogger niels.horn said...

Thanks for the "crontips" :)
I didn't know about the */5 use for the minutes - It makes life simpler indeed.

October 12, 2009 at 9:21 AM  

Post a Comment

Subscribe to Post Comments [Atom]

<< Home