Wednesday, May 27, 2009

On partitioning your hard drive...

As I've been using Linux for over a decade, I am asked a lot of questions about my favorite operating system. I don't consider myself an expert, just an experienced enthusiast :-)

One question that pops up every now and then is about the "ideal" partitioning scheme for Linux.
The short answer is that there is no such thing as an "ideal partitioning scheme". It all depends on so many things, like:
- your hardware
- how you are going to use your system
- your future needs
- etc...

When a new Linux user asks me how to partition his new drive for Linux, I usually say that he should just use two partitions: one for swap and one for the rest. Yes, even with a 500+ GB drive...
Why? Because new users can get into trouble filling up a partition that was created too small and then things stop working.
Sometimes I get funny reactions like: "But I read that it is better to create a partition for /home, another for /usr/local etc...". And some even show expressions of disbelief...

But, since the question keeps coming back, I decided to write how I partition my system.
Like I said - it all depends... So the partitioning for my situation might not apply to yours, but reading through this post you might learn a thing or two.

First of all, I "don't do windows", so you won't read anything about how to double boot between Linux and Microsoft Windows, but basically you can just create an extra primary partition to waste some part of your hard disk and configure your preferred boot loader (Lilo or Grub) to access the alternative.

What follows is "the long answer" on partitioning your hard drive ;)

1. About types of partitions
Partitions come in two types: Primary and Extended. You can have a maximum of 4 partitions on one hard disk. But one of them can be of the "Extended" type, in which you can create several "Logical" partitions.

Remember this when you create your partitions, as you cannot transform a Primary in an Extended partition without backing up all your data, deleting the primary partition, creating the Extended and Logical partitions and then restoring your data.
After you created four Primary partitions, you're done... No way to go back without erasing!

2. The first partition - SWAP
The SWAP partition was originally invented to use as an overflow for your memory needs. If your system needs more memory than it has available, it starts "swapping" unused data from memory to this SWAP area. This was very common in the days when computers had less memory.
For the readers to have an idea: the first systems I worked with had 1 MB of RAM...
The old rule was that the SWAP partition should be twice the amount of RAM in your system.

Nowadays memory is cheap and plentiful in modern systems, so you might think you won't need a SWAP partition.
But don't just skip right now...
First of all, the SWAP partition is also used to 'hibernate' your system if it is capable of doing this (actually, I think *all* systems are capable, but normally only notebooks use this).
Hibernating simply saves all memory to the hard drive and reloads it when you reboot your system.
Guess where the memory is saved? Yes, on the SWAP partition.
So, despite the old rule, it should be at least the size of your RAM.
If you have like 4 GB of RAM, you won't need 8 GB of SWAP space. A SWAP partition of 4.1 GB is good enough.

If you use several Linux flavors on your system, you can have a single SWAP partition for all of them. Just remember you'll loose the 'hibernate' function if you reboot in a different flavor!

The SWAP partition is of the type "82" for those using fdisk under Linux.

3. The second partition - "root"
The "root" partition is the base (or 'root') of the hierarchical file system. It is mounted as "/" in Linux and from there you can go down the hierarchy to any folder or device.
The standard of how this hierarchy is set up is well defined in the Filesystem Hierarchy Standard (or FHS) for all Unix-like operating systems. You can read more on this standard here and here.
Unfortunately, not all Unix or even Linux flavors adhere 100% to the standard, but that's life...

The root partition is where the basic operating system is installed and all files needed to boot your system, get it connected to the network, etc., should be installed here.
Other files, like data, applications not needed for basic operations, user files, etc., can be on other partitions that are mounted later in the boot process.
The "/root" - the 'home' directory of the "root" user must be on the root partition to guarantee that root can login, even if the network is unavailable or external hard drives are not accessible.

Most modern Linux distributions are about 5 to 7 GB in size when fully installed.
To have some extra space, I usually create my root partitions with a size of 15 GB.

4. Extra "root" partitions
Why create extra root partitions? Because you might want to experiment with newer versions of programs, without messing up your working installation.
Or you might want to try out a completely different distribution of Linux for a change.
I personally keep three "root" partitions on my drive, but two should be enough in most setups.
If you make them 15 GB, like your first root partition, you should be ready for most installations.

5. The "/boot" partition
In Linux the /boot folder is used to store the kernel and the 'initrd' files. It is (normally, but do read on...) only used during the boot process.

Most people forget to make a separate partition for /boot. If you run only one installation of Linux on your system, that's fine. But if you plan to setup a multiple-boot system, with more than one Installation of Linux, possibly mixing different flavors, you might get confused when configuring your boot loader.

For instance, let's say you install Slackware and Ubuntu on your system.
From Slackware, you configure your /etc/lilo.conf to read the kernel from /boot/vmlinuz-xxxx
To include Ubuntu in your lilo menu, you would need define your image as /dev/sdxx/boot/vmlinuz-yyyy
And to make things worse, after booting Ubuntu, your /etc/lilo.conf there will have to be different, referring to the Slackware kernel as /dev/sdyy/boot/vmlinux-xxxx

To keep things simple, have a separate /boot partition on your drive that is accessible from all your Linux installations as /boot. Just take care to name your kernels in an organized way, so that you can remember which kernel belongs to which Linux distro.

Now you will still have several copies of your /etc/lilo.conf, as it stays on the root partition of your Linux installations.
I solved this by creating an /etc folder in /boot, where I put the lilo.conf file and then create a symbolic link in /etc with this command:
# ln -s /boot/etc/lilo.conf /etc/lilo.conf
This way I can edit my lilo configuration from any Linux installation and keep things simple.

As for the size of the /boot partition, I make it 250 MB and that has always been enough for me, even with several kernels from several concurrent installations.

6. The (optional) "/home" partition
The /home partition is the subject of many discussions. Each user on a Unix or Linux system has its own /home/<username> folder, accessible only to this user and the all-powerful root.
User-specific configuration files are stored there as well, mostly in hidden sub-folders like .mozilla, .mc, etc.
If your system is only going to have one user, you might as well store everything you want in your "home" folder. But if your system has several users that share files, it is not a good idea to store them here.

The /home partition also can cause problems if one specific user put a lot of files in his home folder, filling up the whole partition and effectively locking the system for other users that can't even save a simple configuration file. The solution to this problem is configuring a quota-system, but that is beyond the scope of this post.

As a simple rule, I create my /home partition with the size of (number of users) x 2 GB. That should be enough to save all configuration files. Shared data (downloaded MP3 files, pictures from digital cameras, etc.) go in a separate partition for data (just continue reading...)

7. The (optional) "/usr/local" partition
As I wrote above, I create my root partitions with the size of 15 GB. This is normally more than enough for a standard Linux installation plus several extra programs.
If you are installing a server with lots of local, non-standard (read: not provided by the standard installation of your Linux distribution) programs, you might want to create a /usr/local partition.

The size of this partition depends on your specific needs.

8. The (optional) "data" partition
Your desktop (or server) might contain data files you want to share with other users - locally or over the network.
I have seen many discussion on where these files should be saved on a Linux system.
Some prefer the "home" folder of the user, but then they will be available only to that particular user (unless you change the standard permissions, which is not a good idea).
Others prefer a special folder in the root directory, like "/data", "/photos", "/mp3", etc. I personally don't like this idea, as it goes against the FHS mentioned before.

In *my* opinion, data should be stored under /srv/xxxx, which is defined by the FHS as "site-specific data which is served by this system".
If you are sharing your pictures or mp3 files, you are "serving specific data" on your system.
So I mount my "data" partition on /srv/data with sub-folders like "pictures", "music", etc.

The size of this partition is limited by your hard drive only and your specific needs.

9. Final conclusions
Do not forget to define all your partitions in your /etc/fstab file to mount them at boot time.

And - I can't say this enough - it all depends on your needs.
You might not need all these partitions. You can very well survive with just a SWAP and a single root partition on a 1 TB drive. You just might encounter some problems when updating your system...

Labels: , ,

Sunday, December 7, 2008

KDE Network Configuration tool and Slackware

This weekend I tried to help someone on LinuxQuestions.org who asked about the knetworkconf tool from KDEs Control Center.
In Slackware this tool hasn't been working for quite some time, it simply says that our operating system is not supported:


(My KDE is in Portuguese, but you'll get the idea...)

I never bothered about it, as I prefer to do all configuring the 'Slackware-way': editing text files. This way I know what's happening and I am in control.

But I understand that this is not for everyone, so this subject pops up once in a while on the forums and I decided to take a look again.

The knetworkconf tool uses several Perl scripts that are stored in /usr/share/apps/knetworkconf/backends
I am definitely not a Perl-guru, but browsing through the several files, I discovered it was nothing too complicated. Basically, network configuration hasn't changed much in Slackware over the last several versions so first I just added the newer version numbers to all files where they were checked.

Posting the first patch file on LinuxQuestions.org a fellow forum member (thanks Woodsman!) discovered that the KDE tool did not show the configured domain correctly.
This actually was a bug in the tool, not my patch, but I managed to solve it.

After some more testing I discovered that every time I saved any changes with knetworkconf, my Samba server stopped working and was even disabled.
This turned out to be another silly bug in knetworkconf, where they forgot to use a variable that holds the current status of Samba.

I posted all patches on the forum and Pat has already included them in the -current branch (see the changelog).
I also sent the patches upstream to the fellows at KDE, so hopefully it makes it into the next versions there as well.

Those who use Slackware 12.1 (the most recent stable version), can copy the following lines and save them in a file:

*** orig/network-conf 2008-12-06 10:17:03.000000000 -0200
--- /usr/share/apps/knetworkconf/backends/network-conf 2008-12-06 10:25:17.000000000 -0200
***************
*** 78,83 ****
--- 78,84 ----
"vine-3.0", "vine-3.1",
"ark",
"slackware-9.1.0", "slackware-10.0.0", "slackware-10.1.0", "slackware-10.2.0",
+ "slackware-11.0.0", "slackware-12.0.0", "slackware-12.1.0", "slackware-12.2.0",
"gentoo", "vlos-1.2", "freebsd-5", "freebsd-6");

$description =<<"end_of_description;";
*** orig/network.pl 2008-12-06 10:16:52.000000000 -0200
--- /usr/share/apps/knetworkconf/backends/network.pl 2008-12-06 21:52:53.000000000 -0200
***************
*** 541,546 ****
--- 541,550 ----
"slackware-10.0.0" => "redhat-6.2",
"slackware-10.1.0" => "redhat-6.2",
"slackware-10.2.0" => "redhat-6.2",
+ "slackware-11.0.0" => "redhat-6.2",
+ "slackware-12.0.0" => "redhat-6.2",
+ "slackware-12.1.0" => "redhat-6.2",
+ "slackware-12.2.0" => "redhat-6.2",
"gentoo" => "debian-2.2",
"vlos-1.2" => "debian-2.2",
"freebsd-5" => "debian-2.2",
***************
*** 1855,1860 ****
--- 1859,1868 ----
"slackware-10.0.0" => \&gst_network_deb22_get_file,
"slackware-10.1.0" => \&gst_network_deb22_get_file,
"slackware-10.2.0" => \&gst_network_deb22_get_file,
+ "slackware-11.0.0" => \&gst_network_deb22_get_file,
+ "slackware-12.0.0" => \&gst_network_deb22_get_file,
+ "slackware-12.1.0" => \&gst_network_deb22_get_file,
+ "slackware-12.2.0" => \&gst_network_deb22_get_file,
"gentoo" => \&gst_network_deb22_get_file,
"freebsd-5" => \&gst_network_deb22_get_file,
"freebsd-6" => \&gst_network_deb22_get_file,
***************
*** 3985,3990 ****
--- 3993,4002 ----
"slackware-10.0.0" => "",
"slackware-10.1.0" => "",
"slackware-10.2.0" => "",
+ "slackware-11.0.0" => "",
+ "slackware-12.0.0" => "",
+ "slackware-12.1.0" => "",
+ "slackware-12.2.0" => "",
"gentoo" => "",
"vlos-1.2" => "",
"freebsd-5" => "",
***************
*** 4160,4165 ****
--- 4172,4181 ----
"slackware-10.0.0" => "slackware-9.1.0",
"slackware-10.1.0" => "slackware-9.1.0",
"slackware-10.2.0" => "slackware-9.1.0",
+ "slackware-11.0.0" => "slackware-9.1.0",
+ "slackware-12.0.0" => "slackware-9.1.0",
+ "slackware-12.1.0" => "slackware-9.1.0",
+ "slackware-12.2.0" => "slackware-9.1.0",
"gentoo" => "gentoo",
"vlos-1.2" => "gentoo",
"freebsd-5" => "freebsd-5",
***************
*** 4442,4447 ****
--- 4458,4464 ----
[ "hostname", \&gst_parse_fq_hostname, HOSTNAME ],
[ "nameserver", \&gst_parse_split_all_unique_hash_comment, RESOLV_CONF, "nameserver", "[ \t]+" ],
[ "searchdomain", \&gst_parse_split_first_array_unique, RESOLV_CONF, "search", "[ \t]+", "[ \t]+" ],
+ [ "domain", \&gst_parse_fq_domain, HOSTNAME ],
[ "domain", \&gst_parse_split_first_str, RESOLV_CONF, "domain", "[ \t]+" ],
[ "order", \&gst_parse_split_first_array, HOST_CONF, "order", "[ \t]+", ",[ \t]*" ],
[ "hostmatch", \&gst_parse_split_first_bool, HOST_CONF, "multi", "[ \t]+" ],
***************
*** 4591,4596 ****
--- 4608,4617 ----
"slackware-10.0.0" => "slackware-9.1.0",
"slackware-10.1.0" => "slackware-9.1.0",
"slackware-10.2.0" => "slackware-9.1.0",
+ "slackware-11.0.0" => "slackware-9.1.0",
+ "slackware-12.0.0" => "slackware-9.1.0",
+ "slackware-12.1.0" => "slackware-9.1.0",
+ "slackware-12.2.0" => "slackware-9.1.0",
"gentoo" => "gentoo",
"vlos-1.2" => "gentoo",
"freebsd-5" => "freebsd-5",
***************
*** 5439,5444 ****
--- 5460,5469 ----
"slackware-10.0.0" => "slackware-9.1.0",
"slackware-10.1.0" => "slackware-9.1.0",
"slackware-10.2.0" => "slackware-9.1.0",
+ "slackware-11.0.0" => "slackware-9.1.0",
+ "slackware-12.0.0" => "slackware-9.1.0",
+ "slackware-12.1.0" => "slackware-9.1.0",
+ "slackware-12.2.0" => "slackware-9.1.0",
"gentoo" => "gentoo",
"vlos-1.2" => "gentoo",
"freebsd-5" => "freebsd-5",
***************
*** 5719,5725 ****
[ "smbdesc", \&gst_replace_ini, SMB_CONF, "global", "server string" ],
[ "winsserver", \&gst_replace_ini, SMB_CONF, "global", "wins server" ],
[ "winsuse", \&gst_replace_ini_bool, SMB_CONF, "global", "wins support" ],
! [ "smbuse", \&gst_service_bsd_set_status, "/etc/rc.d/rc.samba" ],
[ "interface", \&gst_network_interfaces_set, OLD_HASH ],
[ "gateway", \&gst_network_route_set_default_gw, "%gatewaydev%" ]
]
--- 5744,5750 ----
[ "smbdesc", \&gst_replace_ini, SMB_CONF, "global", "server string" ],
[ "winsserver", \&gst_replace_ini, SMB_CONF, "global", "wins server" ],
[ "winsuse", \&gst_replace_ini_bool, SMB_CONF, "global", "wins support" ],
! [ "smbuse", \&gst_service_bsd_set_status, "/etc/rc.d/rc.samba", "%smbuse%" ],
[ "interface", \&gst_network_interfaces_set, OLD_HASH ],
[ "gateway", \&gst_network_route_set_default_gw, "%gatewaydev%" ]
]
***************
*** 5855,5860 ****
--- 5880,5889 ----
"slackware-10.0.0" => "slackware-9.1.0",
"slackware-10.1.0" => "slackware-9.1.0",
"slackware-10.2.0" => "slackware-9.1.0",
+ "slackware-11.0.0" => "slackware-9.1.0",
+ "slackware-12.0.0" => "slackware-9.1.0",
+ "slackware-12.1.0" => "slackware-9.1.0",
+ "slackware-12.2.0" => "slackware-9.1.0",
"gentoo" => "gentoo",
"vlos-1.2" => "gentoo",
"freebsd-5" => "freebsd-5",
*** orig/platform.pl 2008-12-06 10:17:58.000000000 -0200
--- /usr/share/apps/knetworkconf/backends/platform.pl 2008-12-06 10:35:27.000000000 -0200
***************
*** 90,95 ****
--- 90,99 ----
"slackware-10.0.0" => "Slackware 10.0.0",
"slackware-10.1.0" => "Slackware 10.1.0",
"slackware-10.2.0" => "Slackware 10.2.0",
+ "slackware-11.0.0" => "Slackware 11.0.0",
+ "slackware-12.0.0" => "Slackware 12.0.0",
+ "slackware-12.1.0" => "Slackware 12.1.0",
+ "slackware-12.2.0" => "Slackware 12.2.0",
"freebsd-4" => "FreeBSD 4",
"freebsd-5" => "FreeBSD 5",
"freebsd-6" => "FreeBSD 6",
*** orig/service.pl 2008-12-06 10:16:37.000000000 -0200
--- /usr/share/apps/knetworkconf/backends/service.pl 2008-12-06 10:36:52.000000000 -0200
***************
*** 275,280 ****
--- 275,284 ----
"slackware-10.0.0" => "slackware-9.1.0",
"slackware-10.1.0" => "slackware-9.1.0",
"slackware-10.2.0" => "slackware-9.1.0",
+ "slackware-11.0.0" => "slackware-9.1.0",
+ "slackware-12.0.0" => "slackware-9.1.0",
+ "slackware-12.1.0" => "slackware-9.1.0",
+ "slackware-12.2.0" => "slackware-9.1.0",

"gentoo" => "gentoo",
"vlos-1.2" => "gentoo",

Be careful with the longer lines!
Next, open a console as root and type the following:

cd /usr/share/apps/knetworkconf/backends
patch < (your_saved_patchfile)

This should work with all versions since 11.0, but I only tested it on 12.1 and 12.2-RC1, so all standard disclaimers apply...

The result is a working knetworkconf:

Labels: , , ,

Friday, October 3, 2008

Keeping your clock accurate

If you want to be sure that your computer clock is accurate - always - you can configure your system to use NTP (Network Time Protocol).

NTP is one of the oldest protocols on the internet and its predecessors, going back to the early eighties. If you are interested in reading a small article about the history of NTP, I recommend this text in PDF. The official 'home' of NTP is www.ntp.org

You can use NTP on your Linux desktop to automatically synchronize its clock to a pool of servers around the world that are official 'timekeepers'. Usually you configure NTP to talk to about three different servers to guarantee that at least one is available at any time. It is best to choose servers near you, so that response times are shorter, which results in a more accurate clock on your computer. There are more than 1.500 servers to choose from, but some areas on our planet are better served than others. For instance: there are almost a thousand servers in Europe, but only 19 in South America (where I live).

Your first step should be to select three servers close to you - preferably in your country, or if there aren't enough, in your 'zone.
Go to www.pool.ntp.org and browse through the tree of zones and servers to select which ones are best for you.
The 'pooling' system automatically translates the addresses to the actual domain- and server names.
For instance, I use: 0.br.pool.ntp.org, 1.br.pool.ntp.org and 2.br.pool.ntp.org and these names are usually translated to servers at local universities or government institutions.
Don't worry about this part, the pooling system takes care of it.

Now that you have selected your servers, let's start configuring your system. As always, I'll use Slackware as an example, but things will be similar in other distributions.

1. Configure /etc/ntp.conf
Edit this file with your favorite text-editor (I use vi) and go to the lines that say:

server <example-server-name>

and replace this line or lines with your servers. In my case, I put:

server 0.br.pool.ntp.org
server 1.br.pool.ntp.org
server 2.br.pool.ntp.org

While we're editing the configuration, let's make NTP a bit more secure, inserting some restrictions. Find the lines (usually at the end) that start with "restrict ...." and replace them with the following:

restrict default nomodify
restrict 127.0.0.1
restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap nopeer

The address 127.0.0.1 is our local IP address that has no restrictions.
The network / mask combination in the third line is my local network, so that my other desktops can use my system as a time-server, preventing all of them to send NTP queries to the internet.
On the rest of my desktops I put the line "server 192.168.1.1" (my local IP) in their ntp.conf

2. Synchronize your clock manually
If your clock is too much behind or ahead, NTP might refuse to synchronize it, so let's force the first synchronization with a simple command:

ntpdate 0.br.pool.ntp.org

Of course, you should substitute the server name with one from your list.

3. Make your ntp daemon executable
In Slackware this is done with the following command:

chmod +x /etc/rc.d/rc.ntpd

This (again, in Slackware) will make it start every time you boot up your system.

4. Start NTP now without rebooting
Again, a simple command:

/etc/rc.d/rc.ntpd start

That's it!
After a few minutes, check if NTP is communicating with the server pool by running this command:

ntpq -p

It should show the servers it is talking to and one of them should have an '*' in front of it.

Labels: , ,