When people talk about server security, the first things that typically come to mind are technologies like SELinux or applications like Snort. There are such a wide variety of open source security tools that are available that it is almost impossible to keep up with them all, or even to find out about them all.

One tool that deserves mention is OSSEC, an open source, host-based intrusion detection system. It is cross-platform, so it will work on Windows, Linux, and Mac OS X, and it also can employ a client/server architecture that allows you to have one central OSSEC server with a number of monitored OSSEC clients. It provides intrusion detection techniques such as log analysis, rootkit detection, file integrity checking, active response to identified threats, real-time alerting and, for Windows client systems, registry monitoring.

OSSEC may not be the only tool you ever need, but it does consolidate a number of disparate tools into one and, with the client/server model, it allows for consistent monitoring of multiple systems which makes it great for office environments or server farms.

The OSSEC site offers three downloads: the binary client agent for Windows, the OSSEC Web interface, and the source for Unix/Linux systems. Some distributions may provide a binary package for OSSEC, but if they do, ensure it is the latest version. As of this writing, the current version is 1.6.1.

To begin with, download the source for Linux: ossec-hids-1.6.1.tar.gz. It is also a good idea to download the checksum file as well to verify the validity of the file.

# curl -OL http://www.ossec.net/files/ossec-hids-1.6.1.tar.gz
# curl -OL http://www.ossec.net/files/ossec-hids-1.6.1_checksum.txt
# md5sum -c ossec-hids-1.6.1_checksum.txt
# tar xvzf ossec-hids-1.6.1.tar.gz
# cd ossec-hids-1.6.1
# ./install.sh

The above will use curl to download the tarball and checksum file; verify the checksum. If the file verifies okay (ignore failures for non-existent files), unpack it, and execute the install.sh script.

Keep in mind that on Linux you can do a server, agent, or local install. Under the assumption that we only want to monitor the server itself, choose the local install. You must also have gcc installed as the installation will compile the program after asking some questions such as what type of installation, where the files will be located, who to send e-mail to, what options to enable, and so forth. In most cases, the defaults will suffice.

When it comes to the active response questions, the best bet is to take advantage of OSSEC’s ability to add firewall rules to drop potential intruders, rather than using /etc/hosts.deny, which is not nearly as comprehensive; using hosts.deny will only impact services compiled with tcpwrappers support.

Once you have answered the small number of questions, the program will compile. When it is done, you can start OSSEC immediately. By default, the configuration file will be /var/ossec/etc/ossec.conf, and the service itself can be started via the /var/ossec/bin/ossec-control script. Depending on the platform being installed to, you may be asked to manually place OSSEC to start, or it will create an init script to start and stop the service automatically.

The configuration file itself is an XML file, which makes it fairly straightforward to edit. This is where additional log files to be examined would be configured. The OSSEC wiki covers the configuration file in more detail.

Finally, to begin using OSSEC, execute:

# /var/ossec/bin/ossec-control start

For a local configuration, this is all that is required. Further configuration via the ossec.conf file may be desired to further protect the system by specifying additional directives. Installing the web UI may help in this regard, and allow you to see what is going on with the system. The web interface is still in early development, so some functionality is missing, but it may yet prove useful and worthwhile to install and check out.

Delivered each Tuesday, TechRepublic’s free Linux and Open Source newsletter provides tips, articles, and other resources to help you hone your Linux skills. Automatically sign up today!