Installing and mantaining a clean system

Summary:
Learn how to install a Linux system containing only the minimum requirements, and later add software in such way as to maximize performance and reduce runtime errors and version conflicts.

1 Introduction

2 How it's done with package managers

3 How it's done with tarballs (source code)

4 Integrating tarball applications with the system

4.1 Installation

4.2 What does the installation contain?

4.3 How do we integrate the application with the system?

4.4 Application handling, backup and distribution

5 So, what do you propose?

1 Introduction

You may be aware of the fact that on all Unix systems the default directories are there for a reason, since they each serve certain purposes and are organized in a certain way. There are different places for system programs, system daemons, documentation, configuration files, runtime and temporary files, libraries, source code and so on.

As more and more applications are added to a system you are presented with two problems: (1) how to make sure all applications interact perfectly and (2) how to keep track of all the files belonging to a certain application.

Both issues are important. The first one especially, since Unix programs are constantly reusing code either in binary or source form, be it commercial or open source. Thus you have to make sure you don't get overlapping or incorrect versions of the installed software and libraries because this will cause more and more of your applications to stop working partially or altogether.

Keeping track of installed applications will mess up your system just as well over time. If you want to upgrade or uninstall applications you have to know where are the files for each of them, which overlap with similar files from other applications and which you can modify or remove without breaking the others.

In the light of this information, a clean system is a system where you can safely add, remove and update applications at all times, without breaking the functionality of the operating system or other applications in the process.

2 How it's done with package managers

A package manager is an application which is used for the above purpose (handling application maintenance). There are various such managers available: RPM (Red Hat, S.U.S.E., Mandrake), DEB (Debian), PKG (FreeBSD), TGZ (Slackware), SLP (Stampede) and more.

A package contains all the files needed for an application, library or whatever kind of software package it may contain, as well as scripts which perform certain actions upon package installation, removal or update.

The package manager keeps track of installed packages through a central database. All operations are (ideally) done through the manager, who knows how to perform the above actions, make sure no sensible files are removed or overwritten, ensure inter-package dependencies are satisfied and that, generally, all applications interact smoothly.

Advantages of the packages:

• They usually are official distribution releases and have been tested for authenticity and optimal performance.
• They are fast and easy to use.
• They will run on most systems they are meant for.
• Their inter-dependency checking ensured good application interaction.
• Using packages and package managers is recommended for beginners and production systems.

There are also disadvantages to the packages:

• The ready-made binary applications they contain must run on most systems out there, yet there are so many posibilities; therefore, they are compiled against a lower performance standard (the i386 Intel-compatible). They will work on most compatible systems, but will not make use of the more advanced features of the newer hardware.
• There isn't always a package available for your distribution or platform.
• At times you may find yourself in awkward situations: you may need to compile and install an application from source code rather than from a package, and the integration with the package-managed files may not be perfect, leading to problems.
• Package file are piled together (all binaries in one directory, all libraries in another and so on). While these directories respect the standard I mentioned in the introduction, it's still not nice. While you have the package manager to keep track of them it may seem OK, but this practice offers little or no flexibility, in case you need it.

3 How it's done with tarballs (source code)

A tarball is still a package, but of a different kind. First of all, it is traditionally used for shipping the source code for the application as well as various resource-files and the building specifications. They are called tarballs because the traditional packing procedure involves the tar archiver which results in one big file which is later compressed with GZip or BZip2.

The standardized installation procedure for a tarball is the following:

• Unpack the tarball into its own directory under /usr/src.
• Move into that directory.
• Run the ./configure script which will attempt to detect various system specific information and prepare the application for compilation.
• Compile the application by running the make tool; it uses configuration files created by configure for this.
• Provided the compilation works fine make install is usually the final step, and will copy the resulting application to the proper places inside the filesystem.

make is not only used for compilation; it is actually a complex tool used for automating various tasks regarding the application. Its default behaviour is to attempt compilation, but is also used to install, uninstall, test and create default configuration files sometimes.

Ideally, the above steps (./configure && make && make install) are all that's needed for installation. Of course, particular situations may need particular handling:

• configure usually takes parameters, which are used to fine-tune the features and performance of the resulting application.
• The application or library may depend on certain other applications or libraries and will not compile unless they are present.
• Various errors may arise during compilation or installation, most of which can be circumvented with a little tweaking.

So, what can be said about the tarball solution? It sure is slower and more tedious than the package approach. In exchange, the resulting software will offer unique performance tailored specifically to your particular system.

4 Integrating tarball applications with the system

(First of all: I use the term "application" extensively; a tarball may not necessarily contain an application, it can also contain a library or some other software package; thus, the term is generic and should be taken as such.)

The supreme hint is this: in order to keep track of all the files belonging to such an application, you simply install each of them in their own directory, instead of spreading them accross several other directories.

4.1 Installation

The configure script accepts the --prefix parameter which will tell it where the base directory of the future installation is. And on every Unix system there's a directory destined for exactly this task, it's the /opt directory.

So, in order to install, say, xawtv in its own directory you would do this:

$ cd /usr/src
$ tar -xvzf xawtv-X.XX.tar.gz
$ cd xawtv-X.XX/
$ ./configure --prefix=/opt/xawtv
$ make
$ mkdir /opt/xawtv
$ make install

Unless you plan to keep several versions of the same application around, it's a good idea to make an /opt subdirectory named after the app's main name, without version number.

configure also takes other parameters allowing you to specify where subdirectories of the regular installation are placed (in case you don't want them in /opt/app-name).

It's generally a good idea to create a README file under the application's base directory, which contains a brief description of the application, the version, what system it was compiled on, a contact person, a copy of the parameters you used for configure, some tips about how you got things done during installation, and so on. It's not as much trouble as you may think once you turn this into your own standard practice and stick to it. It may save you a lot of trouble later.

4.2 What does the installation contain?

The installation procedure will create certain subdirectories under the application's "home" or "base" directory. Here's a description for their purpose and contents:

• bin - The executable program or programs which form the application.
• lib - Reusable code in binary form, to be used by other applications at run time.
• include - Reusable code in source form, to be used by other applications at compilation time.
• share - The application's own resources (data files, images, etc.)
• man, info, share/doc - Documentation in man, info or other formats (text, HTML, ps, pdf, etc.)

Configuration files are not subject to mandatory placement into the application's base directory (unless they are examples, in which case they usually get bundled within the documentation). The main configuration files will either be placed somewhere under /etc during the installation or will be created when the application itself or its special configuration utility (if any) is ran. There are also secondary configuration files which get placed into the home directory of the user running the application, and are used for future storing of that user's personal preferences.

4.3 How do we integrate the application with the system?

OK, we have installed an application into its own place, but how do we tell the rest of the system it's there? We do it by updating certain system-wide valid configuration files, for each of the installation's subdirectories presented above.

For bin: we need to specify a way to reach the executables from anywhere in the system. This is done via the PATH environment variable. In order to make sure the modification is system-wide we do it in the /etc/profile file by adding this line:

export PATH=$PATH:/opt/app-name

In this example we have concatenated the string "/opt/xawtv" to the already formed PATH and separated them by colons. The stuff in /etc/profile is part of every user environment in the system, and will ensure your application's executables are found. You will need to exit the current shell and re-login in order for the modification to take effect.

For lib: the system keeps track of the binary libraries by looking in the /etc/ld.so.conf file. Simply add /opt/app-name/lib on a line by itself, then run ldconfig to announce the modification.

For include: no system wide settings need to be done. The files in there will be used when a new application needs them for compilation, in which case you can specify their location through parameters passed to configure. For example, GTK needs glib, so its configuration command will look something like this:

./configure --prefix=/opt/gtk --with-glib=/opt/glib

For share: that's none of your business. The application finds and manages the resources under share based on it's starting directory.

For man: similar to the PATH environment variable, only it's called MANPATH and you add it in /etc/man.config like this:

MANPATH /opt/app-name/man

4.4 Application handling, backup and distribution

Placing an application into it's own /opt directory will give you independence from the rest of the system, and a binary application tailored to your system's specifics.

You can pack the application's directory using your favorite archiver and backup the resulting archive on a CD or somewhere else. I usually pack them using tar and bzip2 and create tarballs named like this: appname-version-take-xlife.tar.bz2. By "take" I mean subsequent variations for the same application and the same version. So, for xawtv 3.68 I would get a file called xawtv-3.68-1-xlife.tar.bz2.

The resulting backups can be reinstalled later by simply unpacking them under /opt and adding the necessary modifications to system-wide config files as described above. You can also pass them on to your friends or co-workers with similar systems.

When you need to uninstall you simply delete the application altogether. Take care, however, of the remaining applications' requirements, some of them may depend on the deleted one. I suggest you keep around somewhere a list with which application needs what.

Another good idea: for each application make a nominal listing with all its files and keep it somewhere along with similar listings from other apps. This way, when you delete an application and later another one complains about missing libwhatever.0.0.9.so you look it up and see which application used to provide it.

5 So, what do you propose?

If you're lucky enough to be using a package manager which can do the above tarball-specific footwork and still allow for enough flexibility, by all means use it. In the meantime, you can still get by with a less-than-perfect package manager, especially if you're running a distribution based on it.

For example, I installed my Red Hat 7.1 with only the absolutely necessary RPM packages, things like a shell, basic filesystem and daemons, a compiler and basic network support. No network utilities (like ftp, ssh), no X-Windows or window managers of any kind, no random utilities. After I was done cleaning up I had a RPM-only system consisting of about 130 essential RPM's and a total installation of about 280 Megabytes.

Those initial RPM's are the only RPM's on my machine. Everything else (including X or various libraries and applications) I install and mantain by tarballs. After the initial installation I first put in a better gcc (2.95.3 instead of 2.96 which ships with Red Hat 7.1) and then I started to install tarballs using the new compiler.

It may be slow and tedious work, as I already admitted, but it makes for a very clean and fast resulting system. Plus, this practice is a great way to learn a lot of stuff about Linux.

< Back to the top | Read other columns | Choose what column I should write next! >