Review: Open source OSSEC for host-based intrusion detection

I previously took a look at some of the ways an Intrusion Detection System (IDS) can be deployed to monitor for suspicious network activities. I mentioned host-based Intrusion Detection Systems (HIDS) but didn’t look at any specific examples.

While randomly browsing the software archives, I came across OSSEC HIDS. OSSEC is an open source intrusion detection system that employs log analysis, integrity checking, and rook-kit detection to respond with time-based alerting or active response (the IDS talking action against perceived threats). Being a HIDS, OSSEC should be installed on the system that it is going to monitor. A full installation isn’t always necessary -- if OSSEC is going to be deployed on multiple machines, then a client/server model can be applied. The clients run an agent that sends data back to the server for analysis. Having the ability to monitor multiple systems from one point is pretty much a necessity in a corporate environment and is still quite useful in a home environment.

The thing that most attracted me to OSSEC is the fact that it will run on almost any operating systems including Windows, Linux, OpenBSD/FreeBSD, and MacOS. Actually, only the network agent runs on Windows, which performs all checks except root-kit detection (just where you need it, too!). The OSSEC manual does state that OSSEC doesn’t support Windows root-kit detection "yet" so I wonder whether this feature is in the works?

To test OSSEC out, I’m going to go for a local installation on a fresh Ubuntu 7.04 desktop installation.

The first step is to download the latest release. The manual also recommends verifying its checksum:

# wget

# wget

The sum file includes both MD5 and SHA1 checksums. Here, I came across my first 'issue' which was that the filenames in the sum file did not match up with the OSSEC file name. I edited the sum file to check ossec-hids-latest.tar.gz (rather than ossec-hids-1.2.tar.gz), and the MD5 checked out ‘ok’.

# md5sum –c ossec-hids-latest.tar.gz

ossec-hids-latest.tar.gz: OK

After decompressing and unpacking the archive, I kicked off the install script (./ and waited for the usual mass of errors. Instead, I was prompted to install OSSEC as root -- doh! An 'su –' to root quickly rectified this, and I tried again. Another prompt reminded me that a C compiler must be installed in order for OSSEC to compile. Luckily, I install GCC as default on any desktop Linux installation, as it’s bound to be needed at some point in time. Installation options chosen were:

  • Local installation
  • /var/ossec
  • Yes to email notification: and yes to using my SMTP server
  • Yes to integrity check daemon
  • Yes to rootcheck
  • Active response enabled
  • Firewall-drop response enabled
  • No additions to the whitelist

After this, OSSEC compiled with no errors or warnings at all. The installation script detected Ubuntu and created the correct init script; this must have been a recent improvement, as I had found forum posts with many people complaining that Ubuntu isn’t correctly detected.

So how have I tested OSSEC? So far I have created a new system user, which was detected right away:

Received From: justin-ubuntu->syscheck

Rule: 550 fired (level 7) -> "Integrity checksum changed."

Portion of the log(s):

Integrity checksum changed for: '/etc/passwd'

Size changed from '1504' to '1554'

Old md5sum was: '67ddb6269b9dc8ab219f031d8eb56dde'

New md5sum is : '85335133ce515223c3d4e98f64b00b2f'

Old sha1sum was: '93f6de00e26f4439813694a87decf0c6d93bb5e1'

New sha1sum is : '513f08859b89dd2f36e9ac4be017b92979e116a6'

I also tested the SSHD brute force detection by attempting to authenticate incorrectly. Sure enough, after seven failed attempts, my session was dropped and I had been placed in the hosts.deny file:

Received From: justin-ubuntu->/var/log/auth.log

Rule: 5720 fired (level 10) -> "Multiple SSHD authentication failures."

Portion of the log(s):

Jun 11 19:50:16 justin-ubuntu sshd[7624]: Failed password for root from port 51239 ssh2

Jun 11 19:49:28 justin-ubuntu sshd[7603]: Failed password for root from port 51238 ssh2

Jun 11 19:49:17 justin-ubuntu sshd[7597]: Failed password for root from port 51237 ssh2

Jun 11 19:49:05 justin-ubuntu sshd[7591]: Failed password for root from port 51236 ssh2

Jun 11 19:48:52 justin-ubuntu sshd[7583]: Failed password for root from port 51235 ssh2

Jun 11 19:48:38 justin-ubuntu sshd[7575]: Failed password for root from port 51234 ssh2

Jun 11 19:48:28 justin-ubuntu sshd[7569]: Failed password for root from port 51233 ssh2

Overall, I would say OSSEC seems like a good way for a sysadmin with limited resources to keep tabs on his systems. OSSEC can be used to read Network Intrusion Detection logs (snort) as well as Apache and IIS logs, which make it a great way monitor whether or not your external systems are regularly being checked for vulnerabilities by unknown parties. One issue I will take up with OSSEC is that it seems there is no GUI to ease administration. E-mail alerts or constant monitoring of the logs seem to be the only way to use it. I haven’t found any information on integration with Nagios, Zenoss, or Zabbix, although I am looking. If anyone knows about this or has another suggestion, please let me know!