Most of us have the recommended layers of protection in place.  Firewalls, email filtering, IDS, and IDP systems protect the perimeter and critical network segments.  Hardened servers, anti-malware and carefully managed access controls protect individual devices as deperimeterization increases.  But how effective are these controls?  Is there activity, successful or unsuccessful, that indicates that one or more controls might have failed?  This kind of visibility is provided in large part by security log management.

What is Security Log Management?
Logs containing information relevant to security management are generated by many sources, including:

  • Firewalls
  • Intrusion detection and prevention systems
  • Anti-malware systems, especially centrally managed solutions with aggregated reporting
  • Operating systems
  • Switches
  • Routers
  • Workstations
  • Applications

Many organizations ignore the logs until there’s a security incident.  However, preventive security measures require a daily review of information from business critical sources.  This type of review can help identify (Karen Kent and Murugiah Souppaya, NIST SP 800-92, September 2006):

  • Security incidents
    o Password hacking
    o Large numbers of login failures
    o Malware attacks
    o Port scans
    o Denial of service attacks
    o Excessive errors on network devices
  • Policy violations
  • Fraudulent activities
  • Operational problems
  • Regulatory compliance issues

The challenges to log review can be overwhelming to many businesses.  Logs are continuously growing, are located in many silos, and the staffing and skills necessary to make sense from all the information collected is unavailable.  Security Log Management helps with the process of aggregating, correlating, and reacting to information captured in logs across an enterprise.

Creating a security log management process
The first step in creating a log management process is the creation of a policy.  The policy should define the objectives the organization wants to meet by managing log information.  Supporting standards and guidelines are necessary to ensure policy compliance.  According to Kent and Souppaya, the following issues should be addressed:

  • Log generation
  • Log information transmission
  • Log storage
  • Log analysis
  • Log disposal

When deciding how and when to generate logs, security managers should carefully select the information required.  Data contained in the logs should match those needed to hit management objectives as defined in the policy.  Building a table to list and define log data elements, as in Table 1, can be helpful (Amrit T. Williams, “Define Application Security Log Output Standards”, Gartner ID G00139205, 4 May 2006).

 Table 1

Williams defines additional tables in his paper.

However, all logs are not created equal.  Daily reviews of logs related to an organization’s most critical systems are required.  However, Kent and Souppaya write that less than critical system logs can be on a less stringent schedule.

Finally, a tool should be selected that will pull all the log information into a central place, compare entries from various sources to identify relationships, and provide a portal for viewing the results.  The volume of log information in most organizations makes manual log analysis impractical.  Implementing the right log management solution, whether in-house or from a managed security services provider, is the best way to ensure log analysis provides the best picture of network activity.

Regardless of the solution selected, there are some general best practices that can help ensure success (John Howie, “Security Log Collection”, Windows IT Security, 16 Oct 2006), including:

  1. Secure your log repository.  Log data integrity is important during litigation discovery or when attempting to reconstruct suspected security incidents.
  2. Ensure the clocks on every system from which logs are collected are synchronized.  This is critical for effective event correlation activities.
  3. Frequently revisit the log data elements collected.  As security policies, standards, guidelines and threats evolve, so should your log management process.
  4. Calculate how much storage you’ll need to store log information.  This is closely related to your log disposal policy.
  5. Check the log collection system frequently to ensure it’s working correctly. 

The final word
Log management is an essential part of any security program.  Without the visibility it provides, a security manager lacks the ability to proactively address potential weaknesses in security controls—while reacting blindly to security incidents.