Syslog-NG Installation

Revision as of 06:26, 8 June 2008

The Syslog-NG, or System Log Next-Generation, service is an updated version of the default Syslog service found on FreeBSD and other UNIX and Unix-like operating systems (a paid-for version exists for Microsoft operating systems). Enhancements include the ability to filter content, various methods of storing information including separate files-per-device or MySQL databases and, a key feature, the addition of TCP for transportation of events (typically UDP is used on basic Syslog services).

The source code for Syslog-NG was used to create a commercial product maintained by Balabit under the name Syslog-NG Premium Edition. This is a paid-for-support version of the free Syslog-NG code and revenue from it is used to fund development of the free edition.

Purpose

For further information on the default Syslog service installed on FreeBSD see the main Syslog article. The following is an explanation of configuring Syslog-NG specifically.

The purpose of installing and utilising Syslog-NG becomes apparent when you begin to monitor Syslog events from multiple servers (and other network-attached Syslog-capable devices) and require a centralised method in which to do so. Furthermore the ability to easily track, view and manage Syslog events are important. The enhancements made to Syslog-NG permit this functionality. A separate project called PHP-Syslog-NG exists as a web-based front-end to view and sort events logged to MySQL via the Syslog-NG service. This is also covered here.

Examples

There are a number of reasons why upgrading to Syslog-NG is beneficial:

• a data-centre that houses numerous network devices all capable of sending Syslog event notifications;
• a medium-to-large organisation that has a wide-spread presence in different locations;
• fault detection of unmanned systems;
• other Syslog-capable devices that may not have simplified means to view events.

They all run along similar lines and many medium and larger organisations may benefit from utilising Syslog-NG somewhere on the corporate network.

Installation

The following is a guide to installing (or upgrading to) Syslog-NG on FreeBSD using the MySQL functionality. It also covers the installation and configuration of PHP-Syslog-NG. This is a web-based utility, essentially a set of PHP-based scripts for use within Apache to enable browser access to the Syslog-NG stored events.

Pre-requisites

For reference the following was undertaken by the author on a HP ML570 G2 server using FreeBSD 7.0. The particular release of FreeBSD should not matter too much. In addition the author selected Apache 2.2, MySQL 5.x and PHP 5.x and assigned a second (RAID5) drive entirely to an '/svr' volume (all regular FreeBSD volumes were allocated on the first (RAID1) drive).

For guidance on installing FreeBSD review this article. It is highly recommended, if using MySQL, to allocate either a larger than normal '/var' volume or create a separate volume to store the MySQL database in - for example a secondary drive allocated as '/svr'.

This installation guide will require the following ports to be installed:

• Apache - a full-featured web server, but any PHP-compatible web service should be acceptable;
• MySQL Server - a full-featured database server;
• PHP - a popular scripting language used of web servers;
• PHP-Extensions - a set of extended features for the main PHP language;
• Syslog-NG - the updated Syslog service.

Ensure that the Apache module, MySQL and optionally GD and zlib dependencies are selected under the PHP port options page. The options page is displayed when running make from the ports system. This will install the 'mysql-client' automatically (not to be mistaken with the 'mysql-server' port).

It is recommended that the NTP service is configured to ensure accurate time is maintained against logged events.

The 'PHP-Syslog-NG' web-based utility is not available from the ports system however it can be downloaded from the Google Code website. Extract this into a new folder called '/svr/php-syslog-ng', for example:

# cd /svr
# fetch http:// php-syslog-ng.googlecode.com/files/php-syslog-ng-<version>.tgz
# tar zxvf php-syslog-ng-<version>.tgz
# rm php-syslog-ng-<version>.tgz

Configuration

As with all services that run on FreeBSD they must first be enabled within the rc.conf file. Add the following lines as appropriate:

syslogd_enable="NO"
syslog_ng_enable="YES"
syslogd_program="/usr/local/sbin/syslog-ng"
syslogd_flags=""
mysql_enable="YES"
apache22_enable="YES"
syslog_pipe_enable="YES"

The 'syslog_pipe' entry is a script that is required if using the MySQL functionality and is therefore not part of the other 'syslog' entries.

MySQL

If utilising a larger, separate volume for the MySQL database store the following should be done:

# /usr/local/etc/rc.d/mysql-server stop
# cd /var/db
# mv mysql /svr
# ln -s /svr/mysql mysql

This has the effect of moving the physical 'mysql' database store from the default '/var' volume to the larger '/svr' volume and then creates a symbolic link to the original location. This 'trick' allows MySQL to function without needing to re-configure all of its data store locations individually. For more information on the concepts of symbolic links review this article.

If the MySQL directory does not exist first enable it (see above 'rc.conf' entries) and start MySQL as follows:

# /usr/local/etc/rc.d/mysql-server start

This will create the directory as part of the initial loading phase.

The MySQL server requires a password to be assigned to the 'root' user account (not to be confused with the FreeBSD 'root' user account) which is done using the following command:

# mysqladmin -u root password newpassword

Substitute the 'newpassword' part with a password of your choice.

PHP

The default installation of PHP creates two configuration files one of which requires copying as follows:

# cd /usr/local/etc
# cp php.ini-dist php.ini

The other configuration file is called 'php.ini-recommended' and is more secure for public-facing web server use. Either file can be copied as appropriate for the company security policy, etc.

Open the file 'php.ini' for editing and add the following entry highlighted in bold type:

; The file storage module creates files using mode 600 by default.
; You can change that by using
;
;     session.save_path = "N;MODE;/path"
;
; where MODE is the octal representation of the mode. Note that this
; does not overwrite the process's umask.
;session.save_path = "/tmp"
session.save_path = "/tmp"

; Whether to use cookies.
session.use_cookies = 1

Change the path location as appropriate.

Syslog-NG

In order for Syslog-NG to be aware of what it should log and to where a configuration file must be defined. The configuration is stored under the '/usr/local/etc/syslog-ng' sub-directory. An example called 'syslog-ng.conf.sample' exists and it can simply be copied as 'syslog-ng.conf' to enable Syslog-NG to operate in the same way the default Syslog service would have done. The following is a cut-down version of this file with changes highlighted in bold type to enable network Syslog capability through to the pipe function (which eventually ends up in the PHP-Syslog-NG / MySQL database):

#
# This sample configuration file is essentially equilivent to the stock
# FreeBSD /etc/syslog.conf file.
#
# Additional changes for network Syslog capability
#
# options
#
options { long_hostnames(off); 
	  sync(0);
	  use_dns(yes);
	  use_fqdn(no); };

#
# sources
#
source src { unix-dgram("/var/run/log");
             unix-dgram("/var/run/logpriv" perm(0600));
             internal(); file("/dev/klog"); };

source netsrc { udp(ip("0.0.0.0") port(514));
                tcp(ip("0.0.0.0") port(514)); };

#
# destinations
#
destination messages { file("/var/log/messages"); };
destination security { file("/var/log/security"); };
destination authlog { file("/var/log/auth.log"); };
destination maillog { file("/var/log/maillog"); };
destination lpd-errs { file("/var/log/lpd-errs"); };
destination xferlog { file("/var/log/xferlog"); };
destination cron { file("/var/log/cron"); };
destination debuglog { file("/var/log/debug.log"); };
destination consolelog { file("/var/log/console.log"); };
destination all { file("/var/log/all.log"); };
destination newscrit { file("/var/log/news/news.crit"); };
destination newserr { file("/var/log/news/news.err"); };
destination newsnotice { file("/var/log/news/news.notice"); };
destination slip { file("/var/log/slip.log"); };
destination ppp { file("/var/log/ppp.log"); };
destination console { file("/dev/console"); };
destination allusers { usertty("*"); };
#destination loghost { udp("loghost" port(514)); };
# CISCO Destinations...
destination netlog { file("/var/log/network/$HOST/$YEAR$MONTH$DAY.log" owner(root) group(wheel) perm(0644) create_dirs(yes)); };

destination netsql
               {
               pipe ("/var/log/mysql.pipe"
               template ("INSERT INTO syslog.logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ('$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$ISODATE', '$PROGRAM', '$MESSAGE' );\n")
               template_escape(yes));
               };

#
# log facility filters
#
filter f_auth { facility(auth); };
filter f_authpriv { facility(authpriv); };
filter f_not_authpriv { not facility(authpriv); };
filter f_console { facility(console); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_ftp { facility(ftp); };
filter f_kern { facility(kern); };
filter f_lpr { facility(lpr); };
filter f_mail { facility(mail); };
filter f_news { facility(news); };
filter f_security { facility(security); };
filter f_user { facility(user); };
filter f_uucp { facility(uucp); };
filter f_local0 { facility(local0); };
filter f_local1 { facility(local1); };
filter f_local2 { facility(local2); };
filter f_local3 { facility(local3); };
filter f_local4 { facility(local4); };
filter f_local5 { facility(local5); };
filter f_local6 { facility(local6); };
filter f_local7 { facility(local7); };

#
# log level filters
#
filter f_emerg { level(emerg); };
filter f_alert { level(alert..emerg); };
filter f_crit { level(crit..emerg); };
filter f_err { level(err..emerg); };
filter f_warning { level(warning..emerg); };
filter f_notice { level(notice..emerg); };
filter f_info { level(info..emerg); };
filter f_debug { level(debug..emerg); };
filter f_is_debug { level(debug); };

#
# program filters
#
filter f_ppp { program("ppp"); };
filter f_slip { program("startslip"); };

#
# host filters
#

# CISCO Filters
filter f_netswitch001 {host("10.1.5.1"); };
filter f_netswitch002 {host("10.1.5.2"); };
filter f_netswitch003 {host("10.1.5.3"); };
filter f_netswitch004 {host("10.1.5.4"); };
filter f_netswitch005 {host("172.16.4.1"); };
filter f_netrouter001 {host("10.1.5.9"); };
filter f_netrouter002 {host("172.16.4.2"); };
filter f_netserver001 {host("server1.example.com"); };
filter f_netserver002 {host("server2.example.com"); };
#
# *.err;kern.warning;auth.notice;mail.crit		/dev/console
#
log { source(src); filter(f_err); destination(console); };
log { source(src); filter(f_kern); filter(f_warning); destination(console); };
log { source(src); filter(f_auth); filter(f_notice); destination(console); };
log { source(src); filter(f_mail); filter(f_crit); destination(console); };

#
# *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err	/var/log/messages
#
log { source(src); filter(f_notice); filter(f_not_authpriv); destination(messages); };
log { source(src); filter(f_kern); filter(f_debug); destination(messages); };
log { source(src); filter(f_lpr); filter(f_info); destination(messages); };
log { source(src); filter(f_mail); filter(f_crit); destination(messages); };
log { source(src); filter(f_news); filter(f_err); destination(messages); };

#
# security.*						/var/log/security
#
log { source(src); filter(f_security); destination(security); };

#
# auth.info;authpriv.info				/var/log/auth.log
log { source(src); filter(f_auth); filter(f_info); destination(authlog); };
log { source(src); filter(f_authpriv); filter(f_info); destination(authlog); };

#
# mail.info						/var/log/maillog
#
log { source(src); filter(f_mail); filter(f_info); destination(maillog); };

#
# lpr.info						/var/log/lpd-errs
#
log { source(src); filter(f_lpr); filter(f_info); destination(lpd-errs); };

#
# ftp.info						/var/log/xferlog
#
log { source(src); filter(f_ftp); filter(f_info); destination(xferlog); }; 

#
# cron.*						/var/log/cron
#
log { source(src); filter(f_cron); destination(cron); };

#
# *.=debug						/var/log/debug.log
#
log { source(src); filter(f_is_debug); destination(debuglog); };

#
# *.emerg						*
#
log { source(src); filter(f_emerg); destination(allusers); };

#
# !startslip
# *.*							/var/log/slip.log
#
log { source(src); filter(f_slip); destination(slip); };

#
# !ppp
# *.*							/var/log/ppp.log
#
log { source(src); filter(f_ppp); destination(ppp); };

#
# CISCO Program Filters
#
log { source(netsrc); destination(netlog); };
log { source(netsrc); destination(netsql); };

The majority of the above replicates the original Syslog functionality. The additions send any Syslog events originating from the network to the file-system pipe function. This is subsequently moved from the the pipe to MySQL via another service script.

Apache

The following configuration changes need to be made to Apache. In addition the installation of PHP does not automatically integrate itself into Apache. This is done by manually editing the Apache configuration files. Under Apache 2.2 this has changed from the way Apache 1.3 used to do it.

Under '/usr/loca/etc/apache22/' open 'httpd.conf' for editing and make the following changes highlighted in bold type:

Add the PHP Apache module:

LoadModule userdir_module libexec/apache22/mod_userdir.so
LoadModule alias_module libexec/apache22/mod_alias.so
LoadModule rewrite_module libexec/apache22/mod_rewrite.so
LoadModule php5_module        libexec/apache22/libphp5.so

Also add the default file name for PHP files:

#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
<IfModule dir_module>
    DirectoryIndex index.html index.php
</IfModule>

Near the end of the file add:

NameVirtualHost *:80

Include etc/apache22/Includes/*.conf

Create a configuration file for what will become the PHP-Syslog-NG web interface called 'syslogweb.conf' in the '/usr/local/etc/apache22/Includes/' directory (note the capitalised letter 'i' in 'Includes') containing the following:

<VirtualHost *:80>
   ServerName syslog.example.com
   ServerAlias syslog
   ServerAdmin YourAdminName 

   DocumentRoot /svr/php-syslog-ng/html
   <Directory />
       Options FollowSymLinks
       AllowOverride All
   </Directory>
   <Directory /svr/php-syslog-ng/html>
       # pcw No directory listings
       # Options Indexes FollowSymLinks MultiViews
       Options -Indexes FollowSymLinks MultiViews
       AllowOverride All
       Order allow,deny
       allow from all
   </Directory>

   ErrorLog /var/log/php-syslog-ng-error.log

   # Possible values include: debug, info, notice, warn, error, crit,
   # alert, emerg.
   LogLevel warn

   CustomLog /var/log/php-syslog-ng-access.log combined
   ServerSignature On

</VirtualHost>

Taking note to change the various 'ServerXxx' entries as appropriate for your network.

Start (or restart) Apache to make the above changes take effect:

# apache start     (or restart)

Ensuring no error messages appear.

PHP-Syslog-NG

The PHP-Syslog-NG web-based utility requires write-access to its directory, which in turn uses the default user and group that Apache uses. This by default is www and www. The following sets the access rights to the PHP-Syslog-NG directory:

# cd /svr
# chown -R www:www php-syslog-ng

This sets the user and group ownership of the 'php-syslog-ng' directory, and the '-R' option (for "recursive") sets all sub-directories as well.

Providing all the above steps have successfully executed the PHP-Syslog-NG web front-end should be available to view. Open a web browser of choice and navigate to the DNS name assigned within the Apache web configuration, such as 'http://syslog.example.com'.

This should result in a page entitled 'pre-installation check' with a series of checks with successful test results in green and unsuccessful (either errors or incorrectly configured options) in red. Some red results may simply be optional components missing, such as the PHP GD component or more severe (as in 'show-stopping') problems with write-access to directories. Correct any issues that have been highlighted (remembering changes to Apache and PHP require an 'apachectl restart' command to be issued).

Click Next and accept the GNU/GPL license, then click Next once more.

The next page entitled 'MySQL database configuration' contains various fields which should be filled in as appropriate. The default passwords provided on the screen are simply 'sysloguser' for the 'sysloguser' user account and 'syslogadmin' for the 'syslogadmin' user account. The user names and passwords should be changed as required. Remember that the 'root' user and password refers to the MySQL user specified above.

Do not specify a 'MySQL Table Prefix', de-select the 'Install Sample Data' option. The 'Install CEMDB Data' option is only necessary if monitoring CISCO Syslog-enabled network equipment. Click Next to save and test these settings.

The next page entitled 'Enter the name of your Php-Syslog-NG site' simply requires a title to be specified. This could be the company name or other appropriate entry. Click Next to continue.

The next page entitled 'Confirm the site URL, path, admin e-mail and file/directory chmods' contains a number of options that have been determined by the installation process. These should largely be correct and require only an email address and password to be specified however specify the options as required. Click Next to continue.

The final page entitled 'Final Step: CEMDB Install' shows a hint to remove the installation directory and option to 'Install CEMDB' if it was selected during the installation process.

Syslog-NG Piping

The key component to joining the Syslog-NG service to the MySQL database store are a few scripts that continuously runs in the background.

This script is called '040.syslog-pipe' and is saved in '/usr/local/etc/rc.d' containing the following:

#!/bin/sh

# PROVIDE: syslog_pipe
# REQUIRE: DAEMON
# BEFORE:  LOGIN
# KEYWORD: nojail

# Code based on documentation found at:
# http://www.freebsd.org/doc/en_US.ISO8859-1/articles/rc-scripting/rcng-daemon-adv.html
#
# Written by DrModiford, www.modiford.com

.  /etc/rc.subr

name="syslog_pipe"
rcvar=`set_rcvar`

command="/usr/local/bin/mysql"
command_args="-u logger --password=l0gg3r syslog < /var/log/mysql.pipe 2>&1 &"

pidfile="/var/run/${name}.pid"

#load_rc_config $name
start_precmd="${name}_precmd"
stop_postcmd="echo Stopped ${name}."

#command="exec /usr/local/bin/syslog-pipe.sh 2>&1 &"
#command_interpreter="/bin/sh"

syslog_pipe_precmd()
{
if [ ! -e /var/log/mysql.pipe ]
then
        mkfifo /var/log/mysql.pipe
fi
return 0
} 

load_rc_config $name
run_rc_command "$1"

This script is called 'syslog-pipe.sh' and is saved in '/usr/local/bin/' containing the following:

#!/bin/sh

# Loop MySQL pipe function
while true
while [ -e /var/log/mysql.pipe ]
do
        /usr/local/bin/mysql -u logger --password=l0gg3r syslog < /var/log/mysql
.pipe
done
sleep 1
done

Ensure the scripts are executable with the following commands:

# chmod 555 /usr/local/etc/rc.d/040.syslog-pipe
# chmod 555 /usr/local/bin/syslog-pipe.sh

The above contains the user name 'logger' and the password 'l0gg3r' which must be granted access to the syslog database on MySQL. This is done using the following commands:

# mysql -u root -p
Enter password:       (enter the MySQL root user password)

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 5.0.51a FreeBSD port: mysql-server-5.0.51a

grant insert, select on syslog.* to logger@localhost identified by 'l0gg3r';

Query OK, 0 rows affected (0.01 sec)

mysql> exit

Final Configuration

The entries in the 'rc.conf' file for the services simply enable them to start, it does not specify the order in which they start. The services start in alpha-numerical order and as such the service start-up scripts need renaming as follows:

# cd /usr/local/etc/rc.d
# mv mysql-server 030.mysql-server
# mv syslog-pipe 040.syslog-pipe
# mv syslog-ng 050.syslog-ng

This ensures that MySQL server is running before the Syslog Pipe process begins piping requests from the Syslog-NG service.

While Unix is well known for never requiring a reboot, the number of changes made make it worth doing.

# reboot       (simple as that!)

Testing

To ensure that Syslog-NG is running in place of the default Syslog service by viewing the process information:

# ps -aux | grep syslog
root      975  0.0  0.1  3124  1424  ??  Ss    8:22PM   0:00.44 /usr/local/sbin/syslog-ng -p /var/run/syslog.pid

Providing 'syslog-ng' is revealed in the list it is safe to assume the upgrade has worked.

The command 'logger' can generate Syslog type events for testing Syslog (and therefore Syslog-NG) services. The following example shows how this can be tested.

# logger -p auth.error "Does it work?"
# tail /var/log/auth.log
Jun  6 20:02:42 servername su: username to root on /dev/ttyp1
Jun  6 20:04:27 servername username : First entry
Jun  6 20:07:11 servername username : First entry
Jun  6 20:15:56 servername sshd[12969]: Accepted keyboard-interactive/pam for username from 10.1.55.16 port 55403 ssh2
Jun  6 20:17:27 servername username : First entry
Jun  6 21:04:51 servername username : Does it work?

This generates an event to the 'auth' logger with level of 'error' using the test notification of 'Does it work?'. This is captured by the Syslog-NG service and directed to the '/var/log/auth.log' log file. The Syslog-NG configuration is set to emulate what the original Syslog service but any Syslog events that arrive via the network will be sent to the MySQL pipe service and redirected to the PHP-Syslog-NG database.

The majority of intelligent network devices and other Syslog-aware servers and services can utilise a remote Syslog server to send event notifications. This is particularly true of CISCO devices using up-to-date IOS operating firmware. An example of the required configuration is as follows:

switch#configure terminal
switch(config)#logging trap debugging
switch(config)#logging 10.1.6.161
switch(config)#exit
switch#copy running-config startup-config      (this can be left until the configuration is finalised)

This configures IOS to log the level 'debugging' and up (which means all events) to the Syslog server on IP address 10.1.6.161.

Typically configuring the terminal and logging into the console of a CISCO device will usually generate an event which should be logged into the MySQL PHP-Syslog-NG table. Logging into the PHP-Syslog-NG web-utility should reveal some events.

If CISCO equipment is not being used then another Unix system can be used to generate some test results. Again, the 'logger' command is used as follows (from a remote server):

# logger -h servername -p local0.notice "Hello World"

Note: when new remote Syslog events are sent to the Syslog-NG server it is sometimes required that the 'Reload Cache' option needs to be selected from the 'Config' page of PHP-Syslog-NG web page in order to be selectable.