Flexible scheduling

As explained in my previous post, I needed a flexible way to schedule a small script to run, with variable parameters.

The basic requirements were:

• simple to configure (no messing in crontabs)


• flexible enough to have different schedules on different weekdays (on Sundays a job should be started at another time than on Mondays, for example)


• should offer the possibility to start different jobs at different times


• I want a log of all commands executed

For instance, I need the command "foo start 1234" to run every day except Sunday at 23:00h, and "foo stop 1234" always at 01:00h.
I also need "bar load 1 2 3" to run every Wednesday at noon and "bar load 4" every other weekday at noon.

And, as I am a Slackware fan, I want to do all configuration by changing a text file. No graphical interface please, as my servers do not even have X installed.

So the challenge was there :)

Using cron
Now I thought of a solution how these scripts should be started or controlled.
The standard crontab (in /var/spool/cron/crontabs/root) could probably control everything I needed, but it would be a cumbersome task to change it every time. But - on the other hand - I did not want to substitute cron, as it does a nice job scheduling simple daily or monthly tasks.

So I decided to use cron to start my own scheduler. As an interval I choose a five minute period, as that would be accurate enough for me. I don't need any job to start at 12:42 exactly, 12:40 or 12:45 if good enough for me.

My script to check all schedules was named "sched5" and I added the following lines to the crontab:

# Special 5-minute-interval job
0 * * * * /usr/local/sbin/sched5 1> /dev/null
5 * * * * /usr/local/sbin/sched5 1> /dev/null
10 * * * * /usr/local/sbin/sched5 1> /dev/null
15 * * * * /usr/local/sbin/sched5 1> /dev/null
20 * * * * /usr/local/sbin/sched5 1> /dev/null
25 * * * * /usr/local/sbin/sched5 1> /dev/null
30 * * * * /usr/local/sbin/sched5 1> /dev/null
35 * * * * /usr/local/sbin/sched5 1> /dev/null
40 * * * * /usr/local/sbin/sched5 1> /dev/null
45 * * * * /usr/local/sbin/sched5 1> /dev/null
50 * * * * /usr/local/sbin/sched5 1> /dev/null
55 * * * * /usr/local/sbin/sched5 1> /dev/null

As can be seen, the script is stored in /usr/local/sbin, in the path for root - as normal users do not need to access it.

The script
The complete script is copied here:

#!/bin/bash
#
# sched5 Check every 5 minutes /etc/sched5.conf
#  for possible jobs to run
#  Needs to be started every 5 minutes from cron
#  Definition of schedules in /etc/sched5.conf
#
# Version: 0.0.1 - Thursday, Jul 23, 2009
#
# Author: Niels Horn (niels.horn@gmail.com)


###################
## Configuration ##
###################

# Standard interval
SCHED5_INTERVAL=5

# Default log
SCHED5_LOG=/var/log/sched5.log

# Default log level:
# 0 = no logging
# 1 = log execution of jobs
# 2 = log everything
SCHED5_LL=1

# Read configuration from /etc/sched5.conf
. /etc/sched5.conf


####################
## Get DAY / TIME ##
####################
DAY=`date +%u`
TIME=`date +%H:%M`
[ $SCHED5_LL -ge 2 ] && \
   echo "Started: Day=$DAY Time=$TIME" >> $SCHED5_LOG


####################################
## Main loop: check all schedules ##
####################################

for sched in `seq $SCHED_NUM`; do

   sched_name=SCHED${sched}_NAME
   eval sched_name=\$$sched_name

   [ $SCHED5_LL -ge 2 ] && \
      echo "-- Schedule $sched = $sched_name" >> $SCHED5_LOG

   # get rules for this schedule
   sched_rules=SCHED${sched}_RULES
   eval sched_rules=\$$sched_rules

   # loop through all rules for this schedule
   cmd_found=0
   for rule in $sched_rules; do

      [ $SCHED5_LL -ge 2 ] && \
         echo "   -- Rule $rule " >> $SCHED5_LOG

      # dissect rule
      rule_day=`echo $rule | cut -f1 -d,`
      rule_tim=`echo $rule | cut -f2 -d,`
      rule_cmd=`echo $rule | cut -f3 -d,`
      # calculate time+interval
      timh=${rule_tim:0:2}
      timm=${rule_tim:3}
      [ "${timm:0:1}" == "0" ] && timm=${timm:1}
      let timm=$timm+$SCHED5_INTERVAL
      [ ${#timm} -eq 1 ] && timm="0"$timm
      tim2=$timh":"$timm

      [ $SCHED5_LL -ge 2 ] && \
         echo "   -- Rule: $rule_day $rule_tim-$tim2 -> $rule_cmd" >> $SCHED5_LOG

      # Check if rule is valid for this day
      if [ "$rule_day" == "*" -o "$rule_day" == "$DAY" ] ; then

         [ $SCHED5_LL -ge 2 ] && \
            echo "      -- day ok" >> $SCHED5_LOG

         # Current time => time of rule?
         if [ "$TIME" == "$rule_tim" -o "$TIME" \> "$rule_tim" ] ; then
    
            [ $SCHED5_LL -ge 2 ] && \
               echo "      -- time start ok" >> $SCHED5_LOG

            # Current time < time of rule + interval?
            if [ "$TIME" \< "$tim2" ] ; then

               [ $SCHED5_LL -ge 2 ] && \
                  echo "      -- time end   ok" >> $SCHED5_LOG
               # ok, we found a command to execute!
               cmd_found=$rule_cmd
            fi
         fi
      fi
   done # all rules for this schedule

   # Did we find a command to execute?
   if [ "$cmd_found" != "0" ] ; then

      [ $SCHED5_LL -ge 2 ] && \
         echo "   -- Starting command $cmd_found" >> $SCHED5_LOG

      # Get command to execute
      cmd=SCHED${sched}_CMD_$cmd_found
      eval cmd=\$$cmd

      [ $SCHED5_LL -ge 1 ] && \
         echo "$TIME Schedule $sched / $sched_name: Starting command \"${cmd}\"" >> $SCHED5_LOG

      # Execute command
      . $cmd

   fi

done # all schedules

[ $SCHED5_LL -ge 2 ] && \
   echo "Finished" >> $SCHED5_LOG

# all done!

Save this script as /usr/local/sbin/sched5 and chmod it to 700, as only root needs to read and execute it. It can also be downloaded from my site.

The configuration file
The configuration file is saved as /etc/sched5.conf and is a simple text file.
To explain to options, I put the examples I gave in the beginning of this post in this configuration:

#!/bin/bash
### Configuration for 'sched5' ##############################################
##
## requires sched5 to be started every 5 minutes from crontab
##

#########################
## Standard Parameters ##
#########################

### Location of log
# Default = /var/log/sched5.log
#SCHED5_LOG=/var/log/sched5.log

### Logging Level
# 0 = log nothing
# 1 = log execution of jobs
# 2 = log everything
# Default = 1
#SCHED5_LL=2


###############
## Schedules ##
###############

# Number of defined schedules
SCHED_NUM=2

##
## Start of Schedule 1 (SCHED1)
## Repeat following lines for each schedule
## Changing the variables from 'SCHED1' to 'SCHED2', 'SCHED3', etc.
##

# Name of schedule, only used in log
SCHED1_NAME=foo_schedule

# Commands to execute
# valid commands are SCHEDx_CMD_1 to SCHEDx_CMD_9
SCHED1_CMD_1='foo start 1234'
SCHED1_CMD_2='foo stop 1234'

# Rules for execution: day,time,cmd
# where day = day to execute (1=Mo,2=Tu,...7=Su, *=all)
# time = time to execute command
# cmd = # of command to execute (1-9, 0=do nothing)
# Separate rules with space, later rule has preference over earlier rule,
# so you can start with a general rule for all days (*), then alter for
# specific days (like 7=Sunday)
# Use cmd=0 to cancel a more general rule (like day=*)
SCHED1_RULES="\
*,23:00,1 \
7,23:00,0 \
*,01:00,2"

##
## End of Schedule 1 (SCHED1)
##

SCHED2_NAME=bar_schedule
SCHED2_CMD_1='bar load 1 2 3'
SCHED2_CMD_2='bar load 4'
SCHED2_RULES="\
*,12:00,2 \
3,12:00,1"


Dissecting the configuration file
The first lines define the level of logging and where the log should be saved.
The defaults are explained in the text.

In this example we have two different schedules:

# Number of defined schedules
SCHED_NUM=2


The first schedule has more comments, to explain all options.

First we see the name of the schedule, only used in the log:

# Name of schedule, only used in log
SCHED1_NAME=foo_schedule


Then we'll find the several commands we can schedule:

# Commands to execute
# valid commands are SCHEDx_CMD_1 to SCHEDx_CMD_9
SCHED1_CMD_1='foo start 1234'
SCHED1_CMD_2='foo stop 1234'

All commands must be executable from the command line by root to function.
We can actually have several different commands, and to be exact, names can also end in words, like "SCHED1_CMD_start" - just change the code from "1" to "start" in the rules below.

We now go to the rules of this schedule:

# Rules for execution: day,time,cmd
# where day = day to execute (1=Mo,2=Tu,...7=Su, *=all)
# time = time to execute command
# cmd = # of command to execute (1-9, 0=do nothing)
# Separate rules with space, later rule has preference over earlier rule,
# so you can start with a general rule for all days (*), then alter for
# specific days (like 7=Sunday)
# Use cmd=0 to cancel a more general rule (like day=*)
SCHED1_RULES="\
*,23:00,1 \
7,23:00,0 \
*,01:00,2"

As you can see, the special command "0" cancels a previous rule. So in this example, we always run command 1 (foo start 1234) at 23:00h, except on Sundays (day=7).
The second rule is simpler: start command 2 every day at 01:00h.

The second schedule has no comments, to show how simple it is to add extra schedules:

SCHED2_NAME=bar_schedule
SCHED2_CMD_1='bar load 1 2 3'
SCHED2_CMD_2='bar load 4'
SCHED2_RULES="\
*,12:00,2 \
3,12:00,1"


Starting it all
After changing the crontab, saving the script and setting up the configuration, make sure that everything is scheduled by restarting crond:

killall crond
/usr/sbin/crond -l10 >>/var/log/cron 2>&1

This last line comes from /etc/rc.d/rc.M

As the configuration file is re-read every time the script starts, you can just edit the /etc/sched5.conf file and the changes will be in effect immediately.


As always, feel free to comment on this post with questions or suggestions!

Firewall - Blocking a specific IP

This week I needed a solution to block and unblock certain IPs from accessing the internet at certain times.
I didn't want to change my firewall script too much and didn't want to change it every time the rules changed (which change a lot due to variable circumstances).

So I created a two-part solution. In this post I will describe how I changed my firewall and show a simple script that can block or unblock a specific IP address.

In a following post I will describe how I created a script with a configuration file where I can define when to do what.

iptables and chains
Firewalls in Linux are built with iptables and "chains". Check this article for some basics about writing your own firewall script.

In a chain you can put several rules to filter network packets. It is possible to call a chain from another chain. If the called chain does not filter the packet, it automatically returns to the calling chain.

For instance, we have the standard "FORWARD" chain in a firewall.
In the beginning of this chain, we can always call a user-defined chain called "f_ip".

We create the user-defined chain with:

iptables -N f_ip

And pass all forwarded packets through this chain by including this line as one of our first rules in the FORWARD chain:

iptables -A FORWARD -j f_ip

If the "f_ip" chain is empty, or none of the rules filter the packet to be forwarded, the "FORWARD" chain takes over again:



Now I also want to know if a blocked IP address tried to use the internet, so I created another user-defined chain to log & drop packets:

iptables -N f_ip_drop
iptables -A f_ip_drop -j LOG --log-level 6 --log-prefix "FW: ip: "
iptables -A f_ip_drop -j DROP

Take note that for the time being, nothing goes to the f_ip_drop chain!

But now I can use an external script to add rules to the f_ip chain to filter a specific address without having to change my firewall script.
This external script could contain something like:

iptables -A f_ip -s a.b.c.d -j f_ip_drop

(where "a.b.c.d" is substituted by a real IP address of course)

This part of the firewall then works as follows:



The external script
So now I had my firewall altered and I was ready to write my script to block and unblock specific IP addresses.

The result is here:

#!/bin/bash
#
# fw_ipfilt Filter specific IP addresses
# Needs rc.firewall 0.2.3 or newer
#
# Version: 0.0.1 - Thursday, Jul 23, 2009
#
# Author: Niels Horn (niels.horn@gmail.com)


###################
## Configuration ##
###################

IPT=iptables


######################
## Clear IP filters ##
######################
ip_clear() {
  # flush f_ip chain
  $IPT -F f_ip
}


######################
## Add IP to filter ##
######################
ip_addip() {
  ip=$1
  # send all packets from or to $ip to f_ip_drop chain
  $IPT -A f_ip -s $ip -j f_ip_drop
  $IPT -A f_ip -d $ip -j f_ip_drop
}


###########################
## Remove IP from filter ##
###########################
ip_delip() {
  ip=$1
  # delete rules for $ip from f_ip chain
  $IPT -D f_ip -s $ip -j f_ip_drop
  $IPT -D f_ip -d $ip -j f_ip_drop
}


###################
## Check Command ##
###################

case "$1" in
  'add')
    ip_addip $2
    ;;
  'clear')
    ip_clear
    ;;
  'del')
    ip_delip $2
    ;;
  *)
    echo "use $0 add|del <ip> or $0 clear"
esac

I saved this script as /usr/local/sbin/fw_ipfilt and made it executable and readable only for root (chmod 700).
Now I can block an IP address at any time with:

fw_ipfilt add 192.168.1.123

and unblock it with>

fw_ipfilt del 192.168.1.123

To unblock all previously blocked addresses use:

fw_ipfilt clear



As always, if you have any questions or suggestions, feel free to comment on this post!