One of the great advantages of Linux is its ability to run happily on yesterday's hardware, unlike some other resource-hogging operating systems we all love to hate. Finding a server to run a basic Linux/Apache setup is often as easy as looking around for whatever unused old computer you have laying around. Many a system administrator still contentedly churns out web content on her old Pentium 166 from 1997. Sometimes, however, if you're lucky, your web site will generate enough traffic that it will outgrow such a setup. When apache begins to consistently cause excessive system loads, it's time to think about an upgrade. But never fear - you needn't break the bank for enhanced performance. With just a little more elbow grease, you can set up a small cluster, easily capable of meeting your increased demands.

One of the great advantages of Linux is its ability to run happily on yesterday's hardware, unlike some other resource-hogging operating systems we all love to hate. Finding a server to run a basic Linux/Apache setup is often as easy as looking around for whatever unused old computer you have laying around. Many a system administrator still contentedly churns out web content on her old Pentium 166 from 1997. Sometimes, however, if you're lucky, your web site will generate enough traffic that it will outgrow such a setup. When apache begins to consistently cause excessive system loads, it's time to think about an upgrade. But never fear - you needn't break the bank for enhanced performance. With just a little more elbow grease, you can set up a small cluster, easily capable of meeting your increased demands.

 

One of the beautiful things about a load-balanced cluster is the way in which it scales. When your old P-166 isn't doing the job anymore, add another P-166. When 2 ain't good enough, add a third. It might go against everything you've learned, but when it comes to clusters, it's quantity - not quality. And the great thing is, not only does it work, it works really, really well. This is why Google runs on warehouses full of old PC's rather than one super huge supercomputer.

Overview

Let's get down to the details of how it works. In a traditional (non-clustering) server setup, the web files reside on a single server that serves them up as requested. Simple. In a clustering setup, a web request will direct a client to one of multiple web servers, called nodes.  Each and every one of these nodes needs to have access to a copy of the web files. One way to accomplish this would be to put a duplicate copy of your web files on every computer. A better way, however, would be to have all the files reside on one server (either one of the cluster nodes or, preferably, another server designated as a file server). Each of the nodes can then use NFS to mount the directory containing the web files. This eliminates the needs to maintain multiple copies of all of your web files. 

Because the web requests are spread out over n servers, each server will only be handling 1/n as many requests, which results in less resource usage and a happier server. Another benefit is that if one node is, say, obliterated in a fiery explosion, you simply take its IP address out of the Round Robin DNS, and you're none the worse for wear. You're simply operating with n-1 nodes now, with no effect on the content being served.

Setting up Round-Robin DNS

There are numerous mechanisms for distributing web requests to multiple nodes instead of a single, standalone server. One of the simplest is Round Robin DNS. Normally, when you resolve a domain name, it will resolve to a single IP address. With Round Robin DNS, you make a domain name resolve to two or more IP addresses. The DNS server takes turns sending out each address, so that (in theory), every address will receive an equal amount of requests.

If you're already using BIND (http://www.isc.org/index.pl?/sw/bind/) for DNS for your single-server setup, a Round-Robin setup is easy to configure. You should have an A record that looks something like this:

www.website.com A 172.30.28.101

In order to create a Round-Robin configuration, simple create multiple A
records for the same domain name, like so:

www.website.com A 172.30.28.101

www.website.com A 172.30.28.102

www.website.com A 172.30.28.103

In the above example, 172.30.28.101, 172.30.28.102, and 172.30.28.103 are our 3 cluster nodes. The DNS server will take turns resolving www.website.com to each of the 3. If every DNS lookup resulted in an HTTP request, this would ensure that each of the 3 nodes received an equal number of HTTP requests, thus equally balancing the load of HTTP traffic.  In reality this isn't the case, but Round-Robin DNS still does a reasonably good of distributing traffic - and for it's ease-of-use, it's a popular method of achieving a cluster.

Sharing Web Files Using NFS

As we said earlier, we're going to want to share our web files among all of the nodes in our cluster. Like everything in Linux, the way we can accomplish this are countless. I like NFS because it's easy to set up and does a good job. Before going any futher, you must make sure that your kernel was compiled with support for NFS (compiling it as a module is fine). Next, using your favorite package manager install the NFS server utilities and portmapper software on your file server (or on one of the nodes itself, if you're short on servers). Once these are running, sharing a directory is trivial. All you have to do is create a one-line entry in /etc/exports, and then restart the NFS server. You can read the man page for details on all the options, but here's a typical entry:

/var/shared_www 172.30.28.0/24(rw,sync)

This will allow any host on the 172.30.28.0/24 network to mount this shared directory. The rw option allows clients to have read-write access (as opposed to read-only), and the sync option forces the nfs server to wait until changes have been committed to the disk before responding to reqeusts for the data. This affects performance slightly (but probably unnoticeably), and spares us the risk of data loss and/or corruption in the event of a crash.

For the purpose of this example, let's assume that the NFS server from the above example resides at 172.30.28.104 and we want the shared web directory to be located at /mnt/nfs/shared_www. In order to mount this shared file system, simply install portmapper and the NFS client on each node computer and type the following command:

mount 172.30.28.4:/var/shared_www /mnt/nfs/shared_www

That's all there is to it. If you want to make the change permanent upon reboot, add an entry to /etc/fstabthat looks like this:

172.30.27.4:/var/shared_www /mnt/nfs/shared_www nfs defaults 0 0

Next time you boot, it will automatically get mounted.

Apache Setup

Lastly, you'll need to install and configure apache. This is easy because it's basically identical to setting up apache for a single server, you just need to do it on each node computer. Using all of the exampl information from above, the VirtualHost entry for our web site might look as follows on one of our node servers:

ServerAlias www.website.com

DocumentRoot /mnt/nfs/shared_www

Other Concerns

If your web site uses other data sources for storing information, such as a database or PHP sessions, you'll need to set up a way for each node server to share the information. For a database, this is easy - simply set all of the nodes to connect to the same database server and use the same database (assuming the database solution you're using is a multi-user server solution like MySQL or PostgreSQL).

Sharing PHP sessions are a little more work, but nothing we haven't already covered above. All you need to do is set up another NFS share for the directory where PHP stores session files. I would recommend changing this from /tmp to somewhere else before you do this. There's no reason for all of the cluster nodes to share everything in their /tmp directories.  You can change this with the session.save_path option in your php.ini file (http://us3.php.net/manual/en/ref.session.php#ini.session.save-path). Then just use the techniques mentioned above to create an NFS share for this directory, and mount it on each of the cluster nodes.