Monday, September 29, 2008

Pop-up Dialog in Shell scripts

As I am an old-time computer user, I still prefer doing almost everything from the command line (CLI). I program almost everything in bash-scripts and schedule them with cron.
But of course I do use a graphical interface to read mail (although I have Alpine configured to read my mail from the CLI) and browse the internet (but I also use links a lot!).
Sometimes a bash script needs to give or get feedback, like reporting an error or just telling everything worked fine. Scheduled scripts normally write logs that can be checked later, or can even send e-mails to a user.
There is another, not so well-known option for interaction with the user, if you use KDE as your desktop environment: kdialog
It has many different options to choose from, so I'll give just a few simple examples:

kdialog --title "Hey you..." --yesno "Do you like this popup?"

This pops up the following dialog:


(You'll just have to believe that "Sim" = "Yes" and "Não" = "No" in Portuguese)
The result of which button the user clicked can be obtained by reading the '$?' variable in bash: '0' for the first button (Yes), '1' for the second ('No') and so on.

Another useful example is:

kdialog --title "Warning..." --warningyesno "Low on space on /var - Continue?"

which gives this result:

There are many more options to show radio buttons, check boxes, prompts for passwords, etc.

Now let's use this in a small script:

#!/bin/bash
#
# Sample on how to use kdialog

ERR_SPACE=101

# Calculate needed space
needed=1000 # ok, we're not calculating...

# Calculate space on $dev
dev=/var
space=800

# Test if we have sufficient space left
if [ $space -lt $needed ] ; then
   kdialog --title "Warning..." \
      --warningyesno "Low on space on $dev - Continue?"
   if [ $? -ne 0 ] ; then
      echo "Copy failed" >> log
      exit $ERR_SPACE
   fi
fi

# Copy here...

echo "Copy ok" >> log

# All done

After calling kdialog we test the answer with '$?'. If it is not zero, which means the user clicked on 'no', we exit the script with the defined error code.

For more options, simply enter 'kdialog --help' on the command line.

Happy scripting!

Labels: , ,

Sunday, September 28, 2008

Wonders of the internet

The internet is an amazing thing. Sometimes we forget too fast how things used to be before the internet. Yeah, I know, I'm getting old... I started working with computers when there was no such thing as the internet.

Today I was 'Googling' and by coincidence found a page talking about some software I wrote several years ago. It's a simple Hex-editor for Windows written in C, based on much older versions, dating back from the days of CP/M. If you have no idea what CP/M is, you're probably a bit younger than I am ;-)

This version I found today on the internet dates from 2003 / 2004. I published it at the time for free download (I always believed in free software) on my website. I never promoted it too much, but some people managed to find it and at the time I received quite some feedback.

After finding that page, I did a Google search on this Hex Editor and found pages from all over the world mentioning my software :-) Never had any idea my little program has been used in places like Poland, China, etc...

That's the internet: no more distances and complete anonymity.

Labels: , ,

VMware Problems & Solutions

I am a big fan of VMware and virtualization in general.
At work we use the professional version to install servers that are only used a couple of hours a week, like systems for testing new releases of software etc.
At home I use the free vmware-server-1.0.7 version to test new releases of software, different operating systems, etc. I used to have several desktops for testing, taking up a lot of space. Now I create and delete virtual machines on my desktop that has lots of hard disk space.
After upgradingmy Slackware 12.1-current with the latest versions of libX11 (1.1.5) and gtk2 (2.12.12) VMware stopped working and presented this error:

vmware: xcb_lock.c:77: _XGetXCBBuffer: Assertion `((int) ((xcb_req) - (dpy->request)) >= 0 )' failed.

After some research and testing combinations of the previous versions of libX11 and gtk, I found the following solution:
1) Compile an alternative version of libX11 that does not use xcb
2) Always start VMware using this alternative library

The following script downloads the source code of libX11, compiles it with the '--without-xcb' option and creates a start-up script to use this version with VMware:

#!/bin/bash
#
# vmware_no_xcb Script to compile alternative libX11 library
# (without xcb) for vmware with gtk2 > gtk+2-2.12.9
#
# Version: 1.0.1 - Sun, Sep 21, 2008
#
# Author: Niels Horn
#

# Set constants:
libver=libX11-1.1.5
destlib=/usr/local/bin/vmware_no_xcb
destbin=/usr/local/bin

# Download sources from xorg.freedesktop.org:
test -e $libver.tar.bz2 || wget -c http://xorg.freedesktop.org/archive/individual/lib/$libver.tar.bz2

# Clean old dir and extract sources:
rm -rf $libver
tar xjf $libver.tar.bz2

# Create destination for new lib & new startup script:
mkdir -p $destlib
mkdir -p $destbin

# Go into sources dir, configure, compile & strip:
cd $libver
   ./configure --without-xcb
   make -j 2
   strip src/.libs/libX11.so.6.2.0

# Copy new lib to destination & create symlinks:
   cp src/.libs/libX11.so.6.2.0 $destlib
   ln -s libX11.so.6.2.0 $destlib/libX11.so.6
   ln -s libX11.so.6.2.0 $destlib/libX11.so

# Create startup script, using new lib, and make executable:
   echo -e "LD_PRELOAD=$destlib/libX11.so vmware" > $destbin/vmware-start.sh
   chmod 755 $destbin/vmware-start.sh

# Leave sources dir & remove:
   cd ..
rm -rf $libver

# All done. From now on, use 'vmware-start.sh' to start vmware

Substitute the libver, destlib & destbin variables with your preferences and run this script to do the rest for you.
From then on, start VWware with 'vmware-start.sh' to call your new libX11 library.

This works with the 1.0.x versions of VMware. The new 2.x versions are a completely different story for me, but that's for a future post...

Labels: ,

Saturday, September 27, 2008

'The Man'

Those who use Slackware will know who Patrick Volkerding is. For those who don't: he is the creator and maintainer of the Slackware Linux distribution. Also known as 'The Man'.
Browsing for articles about the history of Slackware (I might write something about it in the future), I found an interview with Pat from 1994 from Linux Journal.
At that time, I was still completely into DOS and Windows (I 'only' discovered Slackware and Linux in 1996). It is fun to read... It even talks about a possible merger between Slackware and Debian (that never happened, as we now know). And there is a rare picture of 'The Man'...
You can find the interview here.

Labels: ,

House-keeping in Linux

Checked your /tmp folder lately? You might be surprised how many files there are if you do not clean it on a regular basis. To keep mine tidy, I have a small script called 'cleantmp':

#!/bin/bash
find /tmp -atime +5 -exec rm -r {} \;
find /var/tmp -type f -atime +30 -exec rm {} \;


Basically it looks for files in /tmp that are at least five days old and removes them, including subdirectories.
In /var/tmp it allows the files to stay a bit longer: 30 days.

To keep your system clean as well, save this script in /usr/local/bin as 'cleantmp' (or any name you prefer), and make it executable with:

chmod +x /usr/local/bin/cleantmp

Now if your system stays on 24 hours a day, you can schedule this script from crontab:

ln -s /usr/local/bin/cleantmp /etc/cron.daily/cleantmp

But if you normally turn of your computer, you can call 'cleantmp' from /etc/rc.d/rc.local

Happy Slacking!

Labels: ,

Friday, September 26, 2008

Traffic Shaping with cbq

Since I am on the subject of networking in Linux, here is another post about some nice things we can do: shaping our traffic so that we can limit how much of our bandwidth we use for each type of traffic.

Just as a small reminder: I am a long-time Slackware user (since 1996) and I only test my configurations on this distribution. I have used other Linux 'flavors' in the past but know much less about them.
Most thing I will post here will work though on other systems, but don't shoot me if they do not.

I started using cbq for traffic shaping on my local network because of the following situation:
I use rsync to copy some files I cannot afford to loose from my desktop to my wife's and v.v. I use crontab to automatically do this at certain hours.
Rsync is a wonderful protocol that only copies files that have changed, saving time and bandwidth.
But sometimes many files are changed or added, and then the whole bandwith of my local network is used, slowing down other traffic.
At these times even browsing the internet can become very slow, just because I am backing up some folders of new digital pictures.

Rsync has its own '--bwlimit' option, but I wanted a better, more structured solution. And this solution is cbq.

Basically configuring up cbq is done in three steps:

1) Setting up cbq
cbq is actually a script that can be found in the documentation of iproute2 in Slackware. We have to copy it to /sbin and make it executable:

cp /usr/doc/iproute2-2.6.16-060323/examples/cbq.init-v0.7.3 /sbin/cbq
chmod +x /sbin/cbq


cbq expects its configuration files in /etc/sysconfig/cbq
If this directory does not exist, create it:

mkdir /etc/sysconfig/cbq

2) Creating the rules-file

cbq reads files in /etc/sysconfig/cbq with the following names:
cbq-nnnn.yyy where:
  • nnnn: is a hexadecimal number from 0002 to ffff
  • yyy: is the name of your network interface, like eth0, eth1, etc

In my case, the network interface for my local network is eth1, so I created "cbq-0002.eth1"
Here is the contents of my file:

DEVICE=eth1,100Mbit,10Mbit
RATE=5000Kbit
WEIGHT=500Kbit
PRIO=5
RULE=192.168.1.110:873,192.168.1.0/24
BOUNDED=no
ISOLATED=no

Some explanations:
  • DEVICE: the interface you want to limit, with its real speed and its weight (1/10 of the max. speed)
  • RATE: the bandwith you want to offer for this particular application / port / address
  • WEIGHT: 1/10 of the RATE
  • PRIO: Priority setting. 5 is default
  • RULE: source,destination --> in my case 192.168.1.110 is my desktop, 873 is the port rsync uses
  • BOUNDED: Default no, used if you have other filters
  • ISOLATED: 'no' means that the rate can be used by other traffic if not in use
3) Starting the bandwidth limiting
Use cbq compile to prepare the new filters or after you alter your cbq-nnnn.yyy files.
Then use cbq start to start your traffic-shaping!
To always start cbq, include it in your rc.local script.

You can monitor your results with iptraf or wireshark.

More information can be found using "man tc-cbq".

Labels: ,

Load Balancing two ISPs

Today I finally managed to use my two ISPs together on my desktop, combining both bandwidths into one big (almost 3Mbit!) pipe.

My setup:
  1. ADSL modem 1Mb down, 320Kb up
  2. GSM modem 2Mb down, 512Kb up
Configuring both at the same time is simple, but then we have two default gateways and our packets always go out through the first one found (or with the lower cost as defined in the 'metric' parameter.)

So how can we divide our packets over both links?

Googling around I found several suggestions to use iptables.
The general idea is:
  • use -m statistic in a chain to choose packets to use on of two links (either the 'nth'-method or the 'average'-method
  • set a mark on the packet
  • use an 'ip rule' to select a routing table for mark 1, mark 2, etc.
That sounded like a perfect solution. This way I could really balance my two links like 40%/60% or whatever.

But it didn't work...

My desktop is not a router, so I have to treat the packets in the OUTPUT chain, where routing has already taken place. The above-described method works on Linux routers treating the PREROUTING chain in iptables, where we can mark a package before routing.

So I studied IP ROUTE and IP RULES a bit more, browsing through the fantastic Linux Advanced Routing & Traffic Control site.

I discovered that we can use 'nexthop' to 'hop' between several routes.
After experimenting a bit I wrote the following script:
#!/bin/bash
#
# bal_local Load-balance internet connection over two local links
#
# Version: 1.0.0 - Fri, Sep 26, 2008
#
# Author: Niels Horn
#

# Set devices:
DEV1=${1-eth0} # default eth0
DEV2=${2-ppp0} # default ppp0

# Get IP addresses of our devices:
ip1=`ifconfig $DEV1 | grep inet | awk '{ print $2 }' | awk -F: '{ print $2 }'`
ip2=`ifconfig $DEV2 | grep inet | awk '{ print $2 }' | awk -F: '{ print $2 }'`

# Get default gateway for our devices:
gw1=`route -n | grep $DEV1 | grep '^0.0.0.0' | awk '{ print $2 }'`
gw2=`route -n | grep $DEV2 | grep '^0.0.0.0' | awk '{ print $2 }'`

echo "$DEV1: IP=$ip1 GW=$gw1"
echo "$DEV2: IP=$ip2 GW=$gw2"

### Definition of routes ###

# Check if tables exists, if not -> create them:
if [ -z "`cat /etc/iproute2/rt_tables | grep '^251'`" ] ; then
   echo "251 rt_dev1" >> /etc/iproute2/rt_tables
fi
if [ -z "`cat /etc/iproute2/rt_tables | grep '^252'`" ] ; then
   echo "252 rt_dev2" >> /etc/iproute2/rt_tables
fi

# Define routing tables:
ip route add default via $gw1 table rt_dev1
ip route add default via $gw2 table rt_dev2

# Create rules:
ip rule add from $ip1 table rt_dev1
ip rule add from $ip2 table rt_dev2

# If we already have a 'nexthop' route, delete it:
if [ ! -z "`ip route show table main | grep 'nexthop'`" ] ; then
   ip route del default scope global
fi

# Balance links based on routes:
ip route add default scope global nexthop via $gw1 dev $DEV1 weight 1 \
   nexthop via $gw2 dev $DEV2 weight 1

# Flush cache table:
ip route flush cache

# All done...


You can download the script here from my homepage.

This is not the perfect solution, as routes are cached, so once you connected to an external site, it will continue to use the linkt hat was originally selected.
So an FTP download won't benefit from this solution, but torrent downloads will, as they use several parallel connections.

I tested the result and managed to download using BitTorrent with the incredible speed of 250KBytes/sec:

Labels: ,

Blogging...

I never wanted to start a blog, for several reasons, but mainly because:
  1. I do not have the time to write regular posts
  2. I really do not have too many interesting facts to share with the world

But it seems that nowadays everyone is blogging all over the world. My wife - who really isn't into computers that much - has a blog and even my son started one.
But hey, I'm self-assured enough to not feel left out yet ;-)
There are other things in life than computers...

I do like to write in forums and newsgroups. I go there to ask questions and to share ideas, solutions, help other people, etc.
Once in a while I write an article about something I found out, a problem I solved or just to receive constructive criticism.

Then some people suggested again I should put these articles in a blog, so that other can find them more easily.

Well, OK, I'll give it a try.

I have lots of stuff here that I wrote down in text files that I might transform into posts in this blog.

Maybe I'll even start enjoying it ;-)