A recurrent theme in many security incidents is that complete information regarding the attacks resides within the logs of the affected organizations, waiting to be discovered and used. Unfortunately the discovery of that information occurs much later, when the damage is already done. Here we will examine how to create a basic log review process that can help in the detection of security incidents. A prompt response might prevent a greater breach when proper review procedures are followed.
In order to successfully implement a log review process, there are some basic requirements that must be put in place:
- A formal logging policy must be established, one that includes all applicable regulatory and operational requirements (such as log review frequency, retention periods, etc.).
- Logging options must be enabled on the systems or devices you wish to monitor. You cannot review logs if you aren't recording any events.
- The time and date of all the systems to be monitored must be synchronized. If any systems reside in different time zones, their time differences must be documented and any log management tools you use must take those time differences into account.
- The responsibility of reviewing logs for a system must not fall only on the person(s) whose actions are being logged in that system.
- The logs themselves must be monitored and protected, and all attempts to terminate or alter the logs must be logged. Such actions could signal a new or ongoing security breach.
Build a baseline
Since the goals of a log review process is to detect unusual or abnormal events, the first step in order to accomplish that is to determine what "normal" is. The exact process for creating a baseline may differ depending on the system being logged and your organization's requirements, but it will usually involve the following:Determine the time period for the baseline
It could be based on events already captured, for example, the last 30 days of events. On the other hand, you could decide that your baseline will be the next 30 days of events logged.Create a report for the baseline events
Once you have captured the events for the time period you selected, you can create a baseline report that includes information such as counts for each message type and average number of events per hour/day/week. The information included in this report should help you describe the normal operation conditions of your systems.
In the process of creating a baseline, you may come across events that require investigation or additional actions. These types of events and their corresponding responses should be documented as they can be used as a starting point in the analysis of new events.
Log review and analysis
The log review process should be performed regularly according to your requirements, regulatory or otherwise, and documented thoroughly in your logging policy. The review basically consists of comparing the new logs produced with the documented baseline. How you perform the comparison depends on the tools at your disposal and the type of systems being logged, among other concerns. Most of the time you simply try to determine whether or not a particular event or sequence of events have been previously logged in either the baseline or the different reviews.
If the new event can be identified as part of the normal operations, it can be then added to the baseline. On the other hand, if during this initial investigation the event does not fit the normal profile, it should be flagged and a more detailed investigation is required.
Depending on the system and the type of flagged event, the investigation may require different approaches. For example, while investigating an event involving a user account it might be necessary to review the timeline of events for that particular user; that way, we might be able to establish a possible context of the particular event. You may also want to determine whether other users have generated the same event or if the event has occurred at another system, either at that time or anywhere else in the past.
During an investigation, it might be necessary to gather information from other sources such as change management systems, anti-malware, and IDS, among others. Another excellent source of information is other people in the organization, as they might provide insight into what the flagged event really is. Lastly, you can ask the application vendor for information on the event, though in some cases this option might not be feasible.
Create an event analysis log
An event analysis log can help you document everything related to the analysis of the events flagged during the regular log review. This log can provide a knowledge base for future investigations; it can be used in conjunction with an incident handling process; or it can be used as evidence of compliance for certain regulations, such as PCI DSS.
An entry in this log should contain:
- The date and time the entry was created.
- Name of the person that created the entry.
- Complete copy of the log entry investigated, including its time stamp and information about the source (such as system name, IP Address, application name, etc.).
- Actions taken during analysis like the ones described previously (for example, looking for other events for the user in the system).
- Conclusions drawn from the analysis, recommended actions, or mitigations (if applicable).
Log review is not just for compliance
Although some regulations mandate that a log review process is in place, all types of unregulated organizations can benefit greatly from establishing one. Hopefully the steps outlined here can be used as basis for a solid log review process that can help detect breaches quickly and reduce the damage done.
I am a technology specialist with over 10 years of experience performing a variety of corporate IT functions, including desktop and server operations, application development, and database administration. My latest role is in information security, focusing on multiple areas including log management and security incident investigation and response.