Main Menu

Members

Centralized Syslog server using RSyslog and PHP-Syslog-NG

Article Stats:

Submitted by: alex

Date: 2009-09-24 00:00:00

Views: 15566

Rating: 4.88

Introduction

This document applies to Debian 5 (Lenny), version 5.0.3 to be more specific. It may work with other versions, but I didn’t test them.

The PHP-Syslog-NG version used is 2.9.9g. In future the new name will be Logzilla. You will see that I’m using the new name during this guide.

How it works:

RSyslog collects all the messages from the various sources and puts them to the MySQL Database.

PHP-Syslog-NG which is the front end interface then looks and reads from the MySQL Database when searched from the web interface.

Some important Notes:

- The guide was tested on Debian 5.0 (Lenny), with RSyslog version 3.18.6

- In order to accomplish what this guide does, you must have an Internet connection.

- All command must be run as “root” user.

Initial Configuration of Debain Linux:

You will have to enable “contrib.” repository in your “/etc/apt/sources.liist” file.

This is how my “sources.list” file looks like:

deb http://mirror.isoc.org.il/pub/debian/ lenny main contrib

deb-src http://mirror.isoc.org.il/pub/debian/ lenny main contrib

deb http://security.debian.org/ lenny/updates main contrib

deb-src http://security.debian.org/ lenny/updates main contrib

deb http://volatile.debian.org/debian-volatile lenny/volatile main

deb-src http://volatile.debian.org/debian-volatile lenny/volatile main

Then update the repository info and upgrade the system to latest packages:

# aptitude update

# aptitude full-upgrade

Install & configure PHP-Syslog-NG (Logzilla):

PHP-Syslog-NG is a front-end for viewing Syslog-ng messages logged to MySQL in real-time, we will make it work with RSyslog also.

It features customized searches based on device, time, priority, message, and date. In the new version authors added a very useful future, an export to Excel!

First, install some packages needed by PHP-Syslog-NG:

# aptitude install apache2 libapache2-mod-php5 php5 php5-cli php5-gd php5-mysql mysql-server msttcorefonts build-essential

You will also need to install the LevenshteinXS perl module in order to use this version PHP-Syslog-NG (Logzilla) .

To install LevenshteinXS from the CPAN archive, type:

# cpan -i Text::LevenshteinXS

Download and extract PHP-Syslog-NG, then remove downloaded file from this directory.

Notice: By the time of writing, the latest version is 2.9.9g.

# cd /usr/share

# wget http://php-syslog-ng.googlecode.com/files/logzilla_v2.9.9g.tgz

# tar xzf logzilla_v2.9.9g.tgz

# mv php-syslog-ng logzilla

# rm logzilla_v2.9.9g.tgz

After running the above commands you should have a new folder in “/user/share/” called “logzilla”.

Configure write permissions for apache:

# chown -R www-data:www-data /usr/share/logzilla/html

Make sure you have PHP memory limit at least 128MB and maximum execution time at least 300 seconds.

Open “/etc/php5/apache2/php.ini”, then…

- Find max_execution_time, set to, at least 300.

- Find memory_limit, set to, at least 128MB.

- Save and exit.

Make log directory:

# mkdir -p /var/log/logzilla

Edit virtual host entry file for apache:

# vi /etc/apache2/sites-available/logzilla

Here is an example of virtual host entry file:

ServerName syslog.planetit.ws

ServerAdmin alex@planetit.ws

DocumentRoot /usr/share/logzilla/html

Options FollowSymLinks

AllowOverride All

</Directory>

# pcw No directory listings

# Options Indexes FollowSymLinks MultiViews

Options -Indexes FollowSymLinks MultiViews

AllowOverride All

Order allow,deny

allow from all

</Directory>

ErrorLog /var/log/apache2/logzilla-error.log

# Possible values include: debug, info, notice, warn, error, crit,

# alert, emerg.

LogLevel warn

CustomLog /var/log/apache2/logzilla-access.log combined

ServerSignature On

</VirtualHost>

Enable apache virtual host:

# a2ensite logzilla

Reload apache:

# invoke-rc.d apache2 reload

Go to web browser for web installation (http://<your_logzilla_url>/install), and then follow the installation screens as described below.

Installation screen 1:

- Make sure everything on this screen is green, if not, fix it before continuing!

- Click “Next >>” at the top right to begin the installation.

Installation screen 2:

- Accept the license agreement and Click “Next >>”.

Installation screen 3:

- Enter the MySQL “root” user password.

- Leave everything else as default unless you really need to change something.

- Click “Next >>” then “OK” to confirm.

Installation screen 4:

- Enter a site name, eg: “PlanetIT Syslog Server”.

- Click “Next >>”.

Installation screen 5:

- Leave the default fields as is, unless necessary.

- Enter email address in the email field.

- Enter a password for the admin or leave the random one there (write it down so you can get into the site).

- Click “Next >>”.

Installation screen 6:

- Click “Install CEMDB”.

Installation screen 7:

- Click “Start import”.

- Wait (be patient, the CEMDB may take some time to populate).

- Click “Continue…”.

Main Site:

- Login using admin and the password you selected on “Installation screen 5”.

If you installed the sample data, there will be a couple of entries, if not, you'll get a message like this:

There appear to be no hosts in the Database yet

You can generate fake ones using scripts/dbgen.pl

Now go back to terminal console and change directory to “/usr/share/logzilla/scripts” and update all files with the correct path for your install using perl:

# cd /usr/share/logzilla/scripts

# perl -i -pe 's/\/var\/www\/logzilla/\/usr\/share\/logzilla/g' *

Open root’s crontab for editing:

# crontab –e -u root

Copy & paste these lines into the crontab:

@daily php /usr/share/logzilla/scripts/logrotate.php >> /var/log/logzilla/logrotate.log

@daily find /usr/share/logzilla/html/jpcache/ -atime 1 -exec rm -f '{}' ';'

*/5 * * * * php /usr/share/logzilla/scripts/reloadcache.php >> /var/log/logzilla/reloadcache.log

Make “logrotate.d” script to work with RSysloag and copy it to logrotate config directory (/etc/logrotate.d):

# cd /usr/share/logzilla/scripts/contrib/system_configs

# perl -i -pe 's/syslog-ng/rsyslog/g' logrotate.d

# cp logrotate.d /etc/logrotate.d/logzilla

Configure RSyslog:

Install MySQL support for RSyslog.

# aptitude install rsyslog-mysql

Select “No” when installation prompts to configure DB for you.

To allow remote UDP connections in RSyslog, open “/etc/rsyslog.conf”, a global configuration file.

- Find #$ModLoad imudpand delete the “#” sign.

- Find #$UDPServerRun 514and delete the “#” sign.

- Save and exit.

Create RSyslog config file for PHP-Syslog-NG (Logzilla):

# vi /etc/rsyslog.d/logzilla.conf

Copy & Paste these lines to the file you just created:

# Write out all log entries to the database formatted for PHP-Syslog-NG

# by Alex Simenduev (http://planetit.ws)

$ModLoad ommysql

$template syslog-ng,"INSERT INTO logs(host, facility, priority, level, tag, program, msg, fo, lo) VALUES('%hostname%', '%syslogfacility-text%', '%syslogseverity-text%', '%syslogseverity-text%', '%syslogtag%', '%syslogtag:R:^[^(\[|:)]*--end:lowercase%', '%msg%', '%timereported:::date-mysql%', '%timereported:::date-mysql%')", SQL

# :ommysql:<db_hostname>,<db_name>,<db_user>,<db_passwd>;<template>

*.* :ommysql:localhost,syslog,syslogadmin,syslogadmin;syslog-ng

Reload RSyslog daemon

# invoke-rc.d rsyslog reload

Configuring Client Machines:

In this example I will explain how to configure Linux machines to send all syslog messages to remote syslog server. It will help you to check the remote sylog intsllation we just did.

This example works on most Linux machines with default syslogd daemon installed.

You must do it with root privileges, (in Ubuntu you can use “sudo” for this).

Edit /etc/syslog.conf:

# vi /etc/syslog.conf

Put on the top of the syslog.conf file you just opened:

*.* @<your_rsyslog_server>

Replace “<your_rsyslog_server>”, with remote RSyslog server IP or hostname.

Restart “syslogd” daemon, for the changes to take effect.

- In RedHat based distributions: # /etc/init.d/syslog restart

- In Debian based distributions: # /etc/init.d/sysklogd restart

Comments

Nov 30th 2009, by Guest
Nice, bit how to setup up the syslogging on the "server"it self? IS it reasonable to delte all local logging config lines and add "*.* @" ? I dont think so :)
I can confirm, the default rsyslog settings on debian/ubuntu do not work with this how to. Lokal logging and remote logging are broken, due to missing "whre to log stuff" cfg settings.

--KI

Dec 3rd 2009, by alex
You don't need to delete any config lines, just add *.* @<your_rsyslog_server> to the top of the syslog config.
---

View my profile

Feb 22nd 2010, by Guest
Make sure to also comment out the the "*.* : oomysql lines" generated by the rsyslog-mysql package installation. In mine it was in /etc/rsyslog.d/mysql.conf

Sep 22nd 2010, by Guest
I am 100% sure this wont work V3.0.91 because
In syslog database for "logs" table there is no field such a "level" and "tag"
Thanks
Joseph John

Sep 26th 2010, by Guest
For V3.0.91 please use this in the lozilla.conf file, this I have tested and is working for me
####
$template syslog-ng,"INSERT INTO logs(host,facility,severity,program,msg,fo,lo) VALUES('%hostname%','%syslogfacility%','%syslogseverity%','%syslogtag:R:^[^(\[|:)]*--end:lowercase %','%msg%','%timereported:::date-mysql%','%timereported:::date-mysql%')", SQL
######

Thanks
Joseph John

Sep 28th 2010, by Guest
I'm trying to install, but when I try to install the web not
work.

the end of the command "cpan-i Text:: LevenshteinXS" appears:
"Warning (Usually harmless): 'YAML' not installed, Will Not store
persistent state "

thereafter, until
"Reload apache:
# Invoke-rc.d apache2 reload
everything works.

when I type "http://192.168.5.2/install" in IE, it says "HTTP 404"

HMoura

Sep 28th 2010, by alex

Sep 28th 2010, by Guest
I'm trying to install, but when I try to install the web not
work.

the end of the command "cpan-i Text:: LevenshteinXS" appears:
"Warning (Usually harmless): 'YAML' not installed, Will Not store
persistent state "

thereafter, until
"Reload apache:
# Invoke-rc.d apache2 reload
everything works.

when I type "http://192.168.5.2/install" in IE, it says "HTTP 404"

HMoura

Couple of questions:
- 192.168.5.2 is the IP of the logzilla, right?
- you did run a2ensite logzilla, right?

Also try to run "invoke-rc.d apache2 restart" and try again
---

View my profile

Sep 28th 2010, by Guest
yes.
yes.

I rebooted, I ran the commands and the problem continues.

"
Last login: Tue Sep 28 12:59:36 2010 from 192.168.1.200
debian-proxy-thunder-srv:~# a2ensite logzilla
Site logzilla already enabled
debian-proxy-thunder-srv:~# invoke-rc.d apache2 restart
Restarting web server: apache2 ... waiting .
debian-proxy-thunder-srv:~#
"

Sep 28th 2010, by alex

Sep 28th 2010, by Guest
yes.
yes.

I rebooted, I ran the commands and the problem continues.

"
Last login: Tue Sep 28 12:59:36 2010 from 192.168.1.200
debian-proxy-thunder-srv:~# a2ensite logzilla
Site logzilla already enabled
debian-proxy-thunder-srv:~# invoke-rc.d apache2 restart
Restarting web server: apache2 ... waiting .
debian-proxy-thunder-srv:~#
"

What you specified in ServerName option in "/etc/apache2/sites-available/logzilla" file?
---

View my profile

Welcome, Guest

Main Menu

Members

Categories

Centralized Syslog server using RSyslog and PHP-Syslog-NG

Comments

Add Comment

	Home \| Contact Us \| About Us
Welcome, Guest	Search: Advanced