Colin Smith explains the differences between Security Information and Event Management systems, log managers, and Microsoft's Audit Collection Services for managing, collecting, and storing event logs for compliance and security purposes.
Many organizations have regulatory compliance reasons for collecting and storing event logs but the value of this information goes well beyond simply passing an audit. Most IT pros and sysadmins are familiar with analyzing the information provided in log files to help find the cause of a problem. Indeed many availability-monitoring tools are based on the premise that information required to pinpoint the root cause of a problem can often be found in the event stream.
Over the last decade, a new breed of event monitoring tools has emerged with a focus on security. There is definitely an overlap between these tools and traditional availability and performance monitoring tools but each category of tool has specific strengths.
Many of our SCOM (System Center Operation Manager) clients are curious about what a relatively new module called Audit Collection Services (ACS) can offer them as it relates to other products that we offer that are more focused on security and log management.
(Full disclosure: At Cistel we have a Canadian Top 10 IT Security practice, and we are an Arcsight partner. I am an Arcsight Certified Professional. We are also a Microsoft Gold Partner with a focus on System Center solutions, so we are well equipped to answer this question.)
In this post, I will attempt to outline the similarities and differences between log managers, SIEMs (Security Information and Event Management) , and Microsoft ACS. In a future post, I will dig a little deeper into the Arcsight SIEM solution.
Log management (LM)
Log management is the basis for many other types of products. Log management is focused on the collection and storage of log files from different types of systems and is not limited to security data. Log management is not normally concerned with the analysis or content of the data. Typically an upstream system will handle the analysis. In practice, many log management tools provide some tools for analysis but rarely reach the level of completeness achieved by SIEM products. Arcsight's Logger fits this role.
Security Information and Event Management (SIEM)
Sometimes referred to as SEM (Security Event Management) or SIM (Security Information Management), SIEM is focused on analysis of security information typically generated from security event logs.
In order to provide the most complete security view, SIEMs generally require data from different types of devices and platforms such as switches, firewalls, routers, servers (Windows, Unix, Linux, etc.) and applications (databases, CRMs, SAP, Exchange, etc.).
This data is then analyzed looking for patterns that are indicative of a specific type of threat like a DoS or a worm. In order to identify these types of attacks, events from different devices need to be correlated. The correlation of events is complicated by the fact that different devices (even from the same manufacturer) produce logs with different formats. Even the data in the logs may have different time and date formats. Even if two devices use the same data format, it is important to synchronize their clocks in order to perform meaningful correlations.
A SIEM should have the ability to normalize multiple event streams so that data formats and time stamps conform to a standardized format before being stored in a database. The normalized data can then be correlated and analyzed with more precision.
SIEM products must either interface with log management products or provide some basic log management functionality built in. Arcsight's ESM interfaces with Logger to provide high end SIEM performance.
Microsoft Audit Collection Services (ACS)
Microsoft Audit collection Services is a component of Microsoft's System Center Operations Manager and requires the SCOM agent to be deployed on systems from which logs will be collected.
MS ACS provides some of the functionality of a log manager and some of the functionality of a SIEM. From a log management perspective, ACS can collect logs in real time from Windows devices. From a SIEM perspective, since ACS is limited to Windows Security event logs the data normalization requirements are minimal. ACS provides the same rich reporting and alerting infrastructure that is part of the Operations Manager product.
For some organizations, MS ACS may be a good fit especially if they already have SCOM deployed. However, there are some limitations such as support for non-Windows devices.
Since ACS does not collect data from network infrastructure devices, it does not include pattern matching algorithms that identify threats like worms and DoS that would typically be found in more complete SIEM solutions.
Some key considerations when choosing a log management and or SIEM solutions for your organization include:
- Data volume and throughput
How many devices do you need to monitor?How many events per second will be generated?
- Storage requirements How much data will be generated on a daily/weekly/monthly/yearly basis?
- Data retention How long will you need to keep the data? Do you have specific regulatory requirements?
- Device & application support What devices and applications do you need the solution to support now and in the future?
- Forensic quality data
Will this data be used for prosecutions?
If so, is there a chain of custody requirement?Is the data guaranteed unalterable?