Avoid these five common IDS implementation errors

Intrusion Detection Systems can go a long way to keep hackers from penetrating your network. However, they can only work if you properly set them up. Here are five common errors and how you can avoid them.

Intrusion detection systems (IDSs) are becoming increasingly important to every company's defense in depth strategy. An IDS acts to alert you that a possible attack is in progress and informs you about characteristics of the attack. In addition to providing information about attacks in progress, an IDS can provide you information about reconnaissance efforts make by intruders in preparation for an attack. A more advanced form of IDS, called IPS (Intrusion Prevention Device) will initiate countermeasures to stop the attack from occurring.

As useful as an IDS/IPS can be in proactively and reactively protecting your network, the IDS will only be as useful and effective as your implementation allows it to be. There is a tremendous amount of confusion in the IDS space today regarding what an IDS should detect, how the IDS should detect, and what an IDS actually is.

There are several common blunders, or implementation errors, that administrators make when setting up their IDS/IPS. These can prevent you from getting the level of protection that you need and expect from your IDS software or device.

Distinguishing between IDS types

In this article, we'll focus on blunders that apply to two different types of IDS approaches:

  • Network IDS (NIDS)
  • Host IDS (HIDS)

A NIDS is a device (often referred to as a "sensor") placed somewhere inside or in front of the corporate network that listens to all traffic moving by the device. A NIDS can be a dedicated hardware device, or it can be implemented as a software add-on to a general purpose network operating system. A pure HIDS doesn't look at the network traffic; instead, the HIDS analyzes log files from a number of system services and network applications running on the host. Some IDS software incorporates functions of both the NIDS and the HIDS.

Both HIDS and NIDS can use one of two primary approaches to intrusion detection: anomaly detection and signature detection.

  • Anomaly detection attempts to profile normal behavior for the environment in which the IDS is placed, and then calls out activity that falls outside this norm. Anomaly detection is akin to what you see with Bayesian anti-spam applications.
  • Signature-based IDS systems include a database of patterns that the IDS matches against the environment on which it's located. Signature-based IDS is similar to what anti-virus applications use to detect viruses on host systems or over the network.

While there are a multiplicity of implementation errors that can be made when deploying an IDS, there are five that we see most commonly. These are:

  • Ignoring frequent false positives
  • Avoiding IPSec to support NIDS
  • Monitoring only inbound connections
  • Using shared network resources to gather NIDS data
  • Trusting IDS analysis to non-expert analysts

In the following sections, we'll take a look at each of these, their ramifications, and how you can be sure to avoid them when you set up your IDS.

Ignoring frequent false positives

Usually, when organizations adopt a new IDS they will turn the IDS system on to detect any and all possible exploits. In other words, they turn the IDS on to the level of its highest sensitivity. This seems like the logical thing to do; after all, you want to catch every intrusion, don't you? Unfortunately, while this configuration enables the IDS to detect a greater number of possible attacks, it also leaves the system open to a greater number of false positives.

False positives are the bane of the IDS world. There are two main errors you'll encounter when working with IDSs: false positives and false negatives. A false positive occurs when the IDS reports a potential attack but there is actually no attack in progress. A false negative is when the IDS fails to report an attack when an attack is in progress. False negatives indicate a failure in the IDS itself, while the false positive shows the IDS is working exactly as it is configured to work.

The problem with false positives is that IDS admins begin receiving hundreds or thousands of alerts per day, overwhelming them and leading them to ignore it when the IDS reports a real attack because it has "cried wolf" so many times in the past. A good example of how this happens is when you've configured your IDS to liberally interpret port scans as attacks. Ports scans are going on all the time, and most of them don't result in actual intrusions. If you configure your IDS to report every port scan, you'll probably soon find that you're ignoring not only the port scan alerts, but any other alert that the IDS sends you. That's human nature--but it can have disastrous results.

You should not ignore false positives or become complacent about them. The IDS is doing its job when it reports an event. You should investigate the events reported to you by the IDS by analyzing the appropriate packet traces or host logs, and also correlate NIDS and HIDS data. If your analysis determines that the IDS is indeed showing too many false positives, then you should dial-down the sensitivity for that alert, or turn it off completely.

Avoiding IPSec to support NIDS

We tend to think of encryption as the ultimate security measure. For example, many network administrators consider VPN connections to be secure connections because the data inside the tunnel is encrypted by MPPE or IPSec. The problem is that many people confuse tunnel encryption with access control. A VPN connection makes the communications between the two VPN endpoints private, but once the data moves outside the VPN endpoints, it is no longer private and thus not secure.

Another common problem with encryption over the wire is that network security devices cannot perform stateful application layer inspection on the contents of the encrypted data stream. This is a major problem in the firewall world, where modern firewalls, such as the ISA Server 2004 firewall, can perform both stateful packet and application layer inspection. The problem is that advanced firewalls can only perform stateful packet inspection (not application layer inspection) on encrypted communications because the firewall does not have access to the encrypted application layer contents; thus, attackers can easily hide their exploits from the firewall's application layer inspection mechanisms by using encrypted tunnels.

As it is with firewalls, so it is with NIDS. The NIDS needs to be able to listen to all traffic moving past its interface and compare that traffic with its rules for legitimate and illegitimate communications. When the NIDS encounters encrypted traffic, the only analysis it can perform is packet level analysis, since the application layer contents are inaccessible. Given that exploits against today's networks are primarily targeted against network services (application layer entities), packet level analysis ends up doing very little to protect our core business assets.

This issue comes into play in environments that have an established NIDS infrastructure and are considering using IPSec for domain isolation. IPSec-based network access controls provide a powerful mechanism to control which hosts are allowed to communication with which hosts on the corporate network. However, one of the key features of both encrypted and non-encrypted IPSec communications is that only the source and destination hosts are able to access the content of the communications.

This creates angst for the "network guys" who run the NIDS on the network, because they believe that IPSec reduces their level of network security since their NIDS devices can't assess the full nature of the IPSec protected communications. While they are correct about hiding the nature of the communications from the NIDS, they are unequivocally incorrect about this reducing the network's overall security posture when you use IPSec. Implementing IPSec for domain isolation can only improve a company's overall security posture.

The solution is to use HIDS in conjunction with NIDS. The HIDS sensors are placed on the IPSec endpoints and can detect potential attacks. The risk of attacks is lower when using IPSec access controls, so the end result is that HIDS data will be easier to interpret because of the lower volume of data. In addition, there will be many fewer false positives.

Monitoring only inbound connections

There are two general approaches to firewalling and access control: control all traffic moving inbound and outbound through the firewall, or control only the traffic moving inbound and allow all outbound traffic to have a "free pass". Unfortunately, even in today's security-conscious world, many organizations don't subscribe to the principle of least privilege and don't enforce outbound access control that enable users and applications Internet access only for those resources they require to accomplish their duties.

The same issues are extant for IDS systems. Some IDS admins are very perspicacious about monitoring external connections, such as DMZs in front of the corporate firewalls, or at the firewall interfaces themselves, but pay no attention to the traffic emanating from corporate network hosts, either outbound to the Internet or to other segments on the corporate network.

Make sure that you place NIDS sensors throughout the organization. The ideal situation is that a NIDS is connected to a spanning port on all network switches. However, if that is cost prohibitive, you should at least place NIDS at the chokepoints on the corporate network. This enables you to monitor outbound and inter-host communications on your network. This is an absolute requirement on today's networks because of the prevalence of network worms and other automated attacks.

Using Shared Network Resources to gather NIDS data

Often NIDS administrators will deploy NIDS sensors on single NIC or multihomed devices on the network that have one or more connections to production network segments. The implication of this configuration is that the NIDS sensor will send data captures over the same interface on which the sensor is sensing.

This is a less than optimal security configuration because the same interface is used to collect data and report it to centralized reporting databases. An attacker can leverage this network configuration to do one of two things:

  • Disable the IDS and prevent it from sending an alert, or
  • Intercept the data being transferred to the reporting database before it reaches the database and change the nature of the data, which could potentially be done with a man in the middle attack.

You can solve this problem by using multihomed NIDS devices and connecting the non-listening interface to a dedicated monitoring network. The dedicated monitoring network connects only the NIDS devices, the reporting database servers, and the monitoring workstations. In the event that the NIDS devices are attacked, an alert can arrive from the monitoring network interface so that you can respond in a timely fashion.

Trusting IDS analysis to non-expert analysts

This one might seem like just a matter of common sense, but it's a situation we see all too frequently. In this scenario, a company has already spent quite a bit of money for NIDS, HIDS or both, and they expect the current staff to be able to implement, manage, and interpret the IDS alerts and comes up with the appropriate responses to these alerts. Often, this belief is based on what the IDS vendor told them in pushing the "turn key" nature of his product.

While the IDS device can do the "heavy lifting" in terms of listening for network traffic, analyzing host log files, and matching the findings against rules configured on the IDS device, an experienced intrusion detection specialist is required to give meaning to many of the events reported by the IDS.

That's because once the potential threat is detected, an intelligent operator must be able to review the alerts for their validity. Many times legitimate communications are interpreted by the IDS as attacks, and if the IDS operators blindly accept these as actual attacks, they could end up denying service to innocent hosts on both internal and external networks.

The IDS analyst must be expert in TCP/IP networking, analysis of networking logs and packet traces, and also extremely well informed about the network services and applications running on the HIDS enabled devices. This level of knowledge is required to correctly interpret the IDS alerts to confirm their validity and then execute the correct intrusion response.