Your log files contain valuable information. Keeping track of them  is a must

You have a PC, you heard about Linux, you decided to try it out, you installed it, you have your Internet connection going, you’ve moved your major applications to Linux, and they work wonderfully. All that was done in a great rush, but now comes the not-so-interesting and rather routine part—managing and administering your system, aka System Administration!

There are many facets to system administration. Having disk space available and keeping data safe are of topmost priority. So let’s talk about them right away. Linux (and as a matter of fact, most flavors of Unix) log various events on the system into files in the /var/ directory. Events could be a system restart, a user login, user logout, an e-mail received or sent, etc. Hence, over a period of time, these log files contain a history of such events. Quite obviously, they eat up a sufficient amount of disk storage. The more you use a system, the bigger these files are. So, for starters, we need to see the available storage on the system. The command df provides a summary of disk space utilization on the various disk partitions. df provides the utilization in terms of 1 kB (1,024 bytes) blocks on the disk. A useful option to use is "-h" that directly provides the summary in terms of megabytes or gigabytes.

 Here’s a typical output:

This listing is from a system that has two hard disks. Linux provides a utility called logrotate that rotates and compresses the system logs, usually found under the directory subtree/var/log.

You can customize logrotate to your liking by editing its configuration files. These are available in /etc/logrotate.d and /etc/logrotate.conf. The script is intuitive and easy to understand. By default, most system logs are rotated four times, uncompressed, before they’re removed from the system. logrotate is typically run once a day from cron. You’ll see an entry similar to the one below in the crontab file:

0 0 * * * /usr/sbin/logrotate

A good start towards minimizing the use of disk storage space would be to uncomment the compress option in /etc/logrotate.conf so that all the rotated log files are kept compressed.

Now, let’s look at the system information contained in the log files. I’ll assign a subjective measure of low, medium and high to each of the files that we mention to indicate their susceptibility to grow fast and occupy a fair amount of disk space (see the table below).

"dmesg" is a file that contains boot-up messages and is perhaps the smallest of the log files. "maillog", as is obvious, contains a log of all incoming and outgoing e-mail. These are created by the message transfer agent (MTA) on the system. Sendmail is the default MTA on Linux, and logs generated by it are logged in maillog. The amount of log information in this file depends on the log level setting in the sendmail configuration file sendmail.cf.

"messages" is a good storehouse of information. The kernel and many other applications that you use are programmed to log their information to this file. The log information in this file is coordinated by a mechanism called syslog (short for system log), with the syslog daemon (syslogd) providing the mechanism on the system. "named" logs its messages in this file, and so does "pppd" when you use it in debug mode. I have a small script, "nuke" that I wrote to kill processes on my system, and this uses syslog to log information in the messages file about the processes it killed.

The "secure" file logs connect and login attempts into your ftp server, as well as failed remote login attempts into your machine. The "wtmp" file provides a record of user logins and their session times, and "last" is a utility that uses this file to provide the data in a readable format. last is typically used to examine the chronological sequence of logins to the system.

Now that you’ve some idea of how system information uses up disk storage, it’s important to prune these files and release disk space. logrotate can be used very effectively to do this. But, it isn’t enough to rotate and throw away the system information. It’s essential to scan the system information at least on a daily basis, to ensure that the system and all applications are working fine. From the system security perspective, it’s an invaluable practice to scan this information. Hence, there is a need to backup these important log files. (Refer to the article Backups and Disaster Recovery in PC Quest, March 1999, page 83)

I’ve touched upon a very small but essential part of system administration here. The amount of system log information generated is proportional to usage, the number of users as well as the applications running. For example, if it’s a personal machine and you use e-mail heavily, you’ll probably have to pay attention to the size of /var/log/maillog.

If as a systems administrator, I were to be granted a wish, I’d wish that future releases of Linux include in them an automated report generator that would give me a report periodically—a summary of the valuable information in all these log files. In my next article, we’ll take a closer look at logrotate.