Octopussy

Introduction
Work in progress...

Octopussy is a log aggregator to combine logs from various sources into a central storage. Octopussy can receive logs from Linux systems, Windows systems and dedicated equipment (routers and such) and make that information searchable. Another reason to centralise logs is to cope with crashing systems: a disk crashing or going corrupt might end the system logs before useful information gets stored, send that same information out over the network and even when a system goes down you will still have the logs.

Another application is to thwart hackers: when a system gets compromised, a skilled hacker will erase any trace of the deed. When logs get erased or (even worse) altered to prevent any sign of malice, an administrator can not find any traces. However, if the logs are sent to a logging server on the network, the logs will still be on the logging server, even after they have been erased from the compromised system.

Octopussy can be seen as the (less powerfull) counterpart of the popular Splunk software. The problem with the latter is the price: 5 digits for a piece of software to analyse and store log information from a small server park is too much for many. As an alternative, Octopussy provides similar features but for free and without any limits, except the ones from your host computer.

Requirements
Octopussy is written mainly in Perl and uses the ASP extension from Apache2 to serve the scripts to your browser.

We will now make sure the basic requirements are installed: * Apache2 * MySQL * Nscd (this is part of glibc so it is installed by default) * Perl (Also a default package) * mod_perl * rrdtool * syslog-ng (this will replace your current logging program) * htmldoc

Start by making sure ITK is enabled for Apache, this allows Apache to switch to a different user for each VirtualHost:

Install everything we need:

Next we need to install a couple of Perl modules, some of which will be already present:

Note that you might need to unmask some packages to install them.

Now we still need some Perl modules but they are not part of the portage tree. So instead, we will use the 'cpan' program to install them:

Type for every module 'install module' like in the example above (when it asks to build modules now, just hit enter): * install Date::Manip * install LWP * install Net:FTP * install Net::LDAP * install Net::SCP * install Proc::PID::File * install Sys::CPU

And finally, install Apache::ASP (the Perl extension used to serve the user interface to a browser):

Installation
After installing all required Perl modules and other programs, it is time to install Octopussy. Grab the latest release (in tar.gz format) from here, currently its version 0.9.9.4.

Extract the file to a subdirectory, in my case I extracted the direct link from SourceForge and used wget:

You now have two options to proceed.

The first one is to use the installer. This has been written to run on Debian and although it looks alright, it tries to restart rsyslog and do some other stuff you most likely won't have.

So that brings us to the second option: do it by hand. In the next section we will explain this method.

Manual Installation
Since the installed will probably fail almost everyone on Gentoo, lets install it by hand instead.

User, group and folder
Create the user and group for Octopussy: or

Create the folders Octopussy will be using:

Copy all files to their destination, this assumes you are in the directory where you extracted Octopussy to:

Symlink AAT to the Octopussy directory:

Change ownership:

MySQL
Now Octopussy is pretty much installed. However, it is missing the database and cron jobs and it might be a good idea to instruct syslog-ng to actually start using Octopussy. Lets start with the database, note that this creates a database called octopussy, with a user octopussy and a password octopussy:

Cron and Init
Lets create the cron job for the logrotate from Octopussy, open '/etc/cron.daily/octo_logrotate' with your favorite editor and put the following in it:

Octopussy comes with its own init scripts, all we need to do it add it to /etc/init.d and modify it a bit to prevent it from starting Apache on its own.

Open '/usr/sbin/octopussy' and comment out the Apache functions on line 263:

... and on line 274:

Now link the script to the init folder so we can add it to the system start up:

...and make it run on startup:

Syslog-ng
Octopussy switched from supporting syslog-ng by default to rsyslog. Since most users will have syslog-ng on Gentoo, we have to modify the configuration ourselves.

Do not forget to restart the logging system to apply the changes:

Apache2
We will now create a special VirtualHost on a separate port for the Octopussy web interface.

Before you continue, be sure to load mod_perl. To do this, edit '/etc/conf.d/apache2' and add '-D PERL' to the parameters:

To test any modifications, you can check the configuration each time before restarting or reloading Apache2:

Lets create a dedicated VirtualHost for Octopussy to use. We will use port 8888 and (for now) we will not use SSL.

Open up your Apache2 configuration and insert the VirtualHost for Octopussy, for example in '/etc/apache2/vhost.d/00_default_vhost.conf':

Logging in
Now start Octopussy and make sure you restarted Apache and Syslog-NG before and then point your browser to http://yourhost:8888 and if all went well you will see the Octopussy login screen.

The default login for Octopussy is admin:admin, it might be a good idea to change that as soon as possible.

Troubleshooting
If you get an error 500 instead of the login screen then something is wrong. This could be incorrect permissions or missing Perl dependencies.

Please look at your /var/log/apache2/error log to get more hints about what is going wrong.

Configuring other devices / systems
After you have verified that everything is peachy, you can modify other systems to forward their logs to Octopussy.

Gentoo Syslog-NG server
Edit the '/etc/syslog-ng/syslog-ng.conf' file and add the following:

destination octopussy_server { udp( "10.0.0.1" port(514) ); }; log { source(src); source(src); destination(octopussy_server); };
 * 1) Send to Octopussy server

Replace 10.0.0.1 with the IP of your Octopussy server.

Once again, this will only work if you have a fairly vanilla syslog-ng installation and do not forget to restart the daemon after modification.

Syslog
Add this to the beginning of your syslog.conf file: *.* @10.0.0.1

Replace 10.0.0.1 with the IP of your Octopussy server.

Restart syslog to apply the changes.

Rsyslog
In case you are using Rsyslog, make sure you append "RSYSLOG_TraditionalFileFormat" as follows:

*.* |/var/spool/octopussy/octo_fifo;RSYSLOG_TraditionalFileFormat

Others
Routers, switches and modems can be set up to forward their logs to Octopussy as well as other logging systems and even Windows.

Look at the Octopussy installation page for more information.