In many situations, having these logs written to a local file system is inconvenient, to say the least. Two specific scenarios come to mind, but perhaps you have others.
The first is, unfortunately, one with which I have personal experience. When attackers compromise a server, they often "clean up" after themselves so that you can't figure out how they broke in. If they're careful, this includes deleting the logfiles so that you can't see what techniques they used. This is particularly true if they used the web server itself as their method of entry, but some scripted exploits simply delete all logfiles for good measure. In this case, it's very handy to have a copy of those logfiles somewhere else.
The second scenario is perhaps less likely, but it is just as interesting. When you have more than one server--particularly when you have dozens or hundreds--it can become quite a chore to look at your logfiles because there are so many of them. In this case, it may be useful to combine them to a central location.
In either of these cases, it is ideal to have the logfiles sent to a central syslog server, rather than having them written to the local file system.
Logging to syslog
As it happens, one thing that very few people know about the Apache web server is that you can log your error log to syslog with a single configuration directive:
ErrorLog syslog:local1
That's almost all there is to it. This tells Apache to send the error log output to the syslog facility called local1. Of course, you also need to tell your syslog server to send this log output off to another server--your syslog server--to write it to a logfile.
Exactly how you do this may vary from one syslog implementation to another. However, the normal Linux syslog uses a file called /etc/syslog.conf (or some variant thereof) to configure how syslog streams.
Because the Apache error log uses syslog-standard severity ratings, you can use normal syslog configurations to split syslog output into different files based on severity. For example, if you wanted to log only critical errors to a particular file, add the line in your /etc/syslog.conf file:
local1.crit /var/log/apache.crit
This still causes the logging of entries of level crit and higher to the file /var/log/apache.crit.
I promised that you could send log entries to another server. Use this line instead:
local1.* @192.168.200.12
You also need to configure the remote syslog server to accept these log entries. For most syslog servers, this involves adding the -r flag to the startup parameters for the syslog server. However, this may vary from one syslog implementation to another, so consult the documentation of whatever particular syslog server you're running.
Streaming Access Logs
Unfortunately, only the error log has this feature built in. It's extremely useful to also have your access logs logged to a remote server, for the reasons described earlier.
There is a technique to allow you to log your access log to syslog as well. However, as with any other article of this nature, I encourage you to check the Apache web server documentation site because this feature may be built-in at some point in the future, rendering this technique obsolete. At the moment, here's what you need to do.
This is a two-step process.
First, create a script that is capable of sending entries to syslog:
#!/usr/bin/perl
use Sys::Syslog qw( :DEFAULT setlogsock );
setlogsock('unix');
openlog('apache', 'cons', 'pid', 'local2');
while ($log =
syslog('notice', $log);
}
closelog
This script, written in Perl, uses the Sys::Syslog module to send data to a syslog server. Make sure that you have the Sys::Syslog module installed, but it should be standard on any fairly recent version of Perl.
Put this script somewhere and make it executable. For this example, I assume that you've put the script at /usr/local/apache/bin/apache_syslog.
Second, point your access log at this script using the piped logfile syntax:
CustomLog |/usr/local/apache/bin/apache_syslog combined
The apache_syslog script will launch when your web server starts up, and it processes logfile entries as it receives them from the server. The while loop will continue for the lifetime of the Apache server process.
As you can see in the script, it sends these entries to a syslog facility named local2, so configure your syslog server with an entry in /etc/syslog.conf like:
local2.* @192.168.200.12
See the documentation for Sys::Syslog for more information on the options that are available to you in this module.
With this configuration, in the event that your server is compromised or there is catastrophic drive failure, you'll still have a remote copy of your logfiles to find out what happened.
Also note that if you have multiple CustomLog directives, you can have multiple logfiles. So, if you want to have a local logfile in addition to a remote one, just add an extra CustomLog directive pointing to a local logfile:
CustomLog /usr/local/apache/logs/access_log combined
Caveats
There are a couple of caveats, of course, to logging to syslog.
First, if you're logging from several servers to a central syslog server, it's important that you keep your clocks synchronized with NTP. If you don't do this, then logfile entries will appear to arrive out or order, and this may cause difficulties when you're trying to post-process the logfiles for statistical information. In particular, some web log-stats software gets bent out of shape if the log entries are out of order, and may refuse to process them.
Second, I noticed that my syslog server will condense several log entries into a single line in the logfile if they're the same. For example, a repeated request for a particular missing file resulted in:
Aug 31 20:18:05 192.168.200.10 last message repeated 19 times
Finally, it is worth mentioning that there are third-party modules available to do logging for multiple servers. The most popular of these modules is mod_log_spread, which will likely be a more efficient solution for very large (many servers) installations.
