Peter@BalaBit

Let’s imagine a webserver with many virtual host. It is jailed to a chroot, but it could be on a virtual machine (even a real one). I do want to collect the logs with the following requirements:

• No messages available by apache
• No need to handle hundres of sources. Transfer every log on one channel (okay let’s use two;-)
• No need to rotate and support archive
• Use secure protocoll, but I guess I do not have to even mention;-)

Les’s see how to start. The first step is force apache to log to a named pipe. Not a big trick, just create the pipes. Use mknod:

mknod /var/log/apache/access.log p

mknod /var/log/apache/error.log p

Now set all the virtualhost to use them:

ErrorLog /var/log/apache2/error.log

LogLevel debug

CustomLog /var/log/apache2/access.log combined

Important, that all the virtual hosts will use these nodes, therefor logs are not stored in the apache chroot any more. Now we have to read messages some way. The only small problem is apache uses Common Log Format, which is fare from any standard syslog format. Fortunateley it is possible to modify it in apache conf. The original looks this:

LogFormat “%h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-Agent}i”" combined

Let’s change it to comply to syslog RFC mine looks this:

LogFormat “Jan 12 12:12:12 %v apache[666]: %h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-Agent}i”" combined

Do not care about strange PRI field and the fixed timestamp neither the funny PID. I only used it because google indexing;-) Which really count is hostname, which continas the name of the virtual host (%v = virutal host) and the original combined message is delivered at the MESSAGE field ($MSGONLY). So now we are ready with the apache side, let’s focus on the syslog-ng side. The first step is reading the two pipes:

source s_apache_access {

pipe(“/var/log/apache2/access.log);

};

On the destination (writin side) we simple sore in different files by hostname field:

destination d_apache_access {

file(“/var/log/apache2/$FULLHOST” template(“$MSGONLYn”) template-escape(no) owner(“root”) group(“adm”) perm(0640));

file(“/var/log/archive/$R_YEAR/apache/$R_MONTH/$FULLHOST.$R_DAY” template(“$MSGONLYn”) template-escape(no) owner(“root”) group(“adm”) perm(0640) create_dirs(yes) dir_owner(“root”) dir_group(“adm”));

};

I like solving archiving under the same time, therefore the second destination does it. Now I have to wire the client and the server side:

destination d_logserver_net {

tcp(“1.2.3.4″ port(514)

tls(ca_dir(“/opt/syslog-ng/etc/syslog-ng/ca.d”)

key_file(“/opt/syslog-ng/etc/syslog-ng/key.d/client.key”)

cert_file(“/opt/syslog-ng/etc/syslog-ng/cert.d/client_cert.pem”)));

};

log {

source(s_apache_access);

destination(d_logserver_net);

};

On the server side we receive the messages:

source s_apache_net {

tcp(ip(0.0.0.0) port(1999)

tls( key_file(“/opt/syslog-ng/etc/syslog-ng/key.d/syslog-ng.key”)

cert_file(“/opt/syslog-ng/etc/syslog-ng/cert.d/syslog-ng.cert”)

ca_dir(“/opt/syslog-ng/etc/syslog-ng/ca.d”)) );

};

log {

source(s_apache);

destination(d_apache_access);

};

We are ready. It was not very difficult, was it?