This weekend I was contacted by a client regarding a server that had run out of room on the main hard drive. Among the things that were corrupted by the drive filling up was the RPM database.

When trying to query of upgrade any package, the error message

A search turned up this howto on repairing the db.

These two commands made quick work of the fix. First remove any old lock files:

Then rebuild the RPM database:

Then copy in any public keys into ~/.ssh/authorized_keys

I have been asked several times recently about my original comparison of Lighttpd and thttpd and subsequent benchmarks for Lighttpd vs thttpd.

I do not have updated benchmarks for these two. However, Lighttpd is still under active development with the latest release Lighttpd 1.4.16 dated 7/24/07. The most recent thttpd release has a newest file date of 6/29/05 (I couldn’t find an official release date for thttpd 2.25b). I would surmise that the performance for Lighttpd would only have increased as it is still under active development. Without a new release, thttpd’s performance surely hasn’t changed since my benchmark.

With the recent increase in SPAM volume, I have had two clients contact me with Qmail problems on Plesk servers. The error they were receiving was in the status page of the Plesk admin panel. Under the status line for Qmail, Plesk said

Several attempts to restart Qmail produced the same results. Restart the server… still no Qmail. Looking through the Qmail settings in Plesk I noticed the server owner had ticked _all_ RBL. Each blacklist check requires a distinct call to the specified RBL service for every email. A busy server with many domains might have several thousand email addresses on it. With only a hundred SPAMs a day to each email address (I get 400+ a day on my main accounts), its easy to see how Qmail could get worked over with a million RBL checks a day. Not to mention the load added to the RBL servers themselves.

I un-ticked all but two RBL checks and Qmail started right up. There is a limit to how many RBL services Plesk Qmail can support. I would recommend only a couple. If many SPAMs are still getting through select a few different ones, don’t just blindly add more checks. More is not always better. Sometimes more is just more. I suggest testing various combinations until you find the best for your server and users.

Another worthwhile note is that these RBLs sometimes go away or are inaccessible for a period of time. If you are trying to use one that is down, all of your mail will be delayed while each attempt has to time out before Qmail can process the mail further. If you have problems with sluggish mail check the sites of the RBLs in use to verify they are indeed active.

These are all useful when trying to track down an open formmail script.

List bounce messages

Freeze bounce messages

Freeze messages from user@domain.com

Find out what user your webserver runs as. Use this as the email address to key on. For example, my Apache runs as nobody so I want to freeze all messages sent from the user nobody@domain.com so I can look through them to see if I can deduce where the insecure formmail script is.

Delete frozen messages

Recursively chmod only directories

Similarly, recursively set the execute bit on every directory

The +X flag sets the execute bit on directories only

Recursively chmod only files

Recursively chmod only PHP files (with extension .php)

Lsof (LiSt Open Files) is a powerful Unix command utility that comes in handy in many security and performance related sysadmin tasks.

List the PIDs of running httpd processes. The -t option specifies terse output, only PIDs.

List process listening to port 80.

List processes of the specified command (-c). Search is implied in this usage. It essentially searches for any process name starting with the specified value. The second example would list all process starting with m, (mysqld, mutt, etc.)

Find processes owned by users apache or nobody.

Find the files using the process specified by PID .

Find processes with files open in the directory specified.

I previously posted a Quick and Dirty MySQL Backup snippet that has gotten a bit of traffic. It works well in a pinch, but it doesn’t have any features. I use it primarily as a safety backup when performing maintenance on a MySQL server.

Several of my clients have requested a script that they can use to do on demand one-off backups. I rolled the snippet up into a very quick-and-dirty MySQL backup.

Download mysql_quick_back.sh

Installation

1. Copy mysql_quick_back.sh to a directory in your path (~/bin/ for me).
2. Set execute bits with
   chmod 755 mysql_quick_back.sh
3. Set MySQL user/pass, directory to save backups to and notification email in script configuration
   # START configure
   # MySQL user, needs SELECT privileges
   # Create a new user with these permissions!  More secure than using a user
   # with full permissions
   user=
   
   # MySQL password
   pass=
   
   # admin email address
   adminemail=
   
   # backup location without trailing slash
   # set to current directory
   backuppath=.
   # or set to your home directory
   #backuppath=/home/admin/db_backs

Usage

Simply execute the script

NOTE

For a full MySQL backup application, check out the original MySQL Backup by Peter Falkenberg Brown. MySQL Backup is robust enough to use as a daily backup. I use it on most of the servers I manage.

The fed mandated daylight savings time changes go into effect this weekend. To ensure you have the updated days do a quick test

If you see this Sunday’s date March 11 2007 then you have the updated package. Sunday’s DST change should be handled.

If you see April 1 2007 your tzdata package needs updating. Upgrade with your favorite package tool up2date, yum, apt-get…

More details at Linux Watch