Rather than waiting for the “you’ve been breached” notification, security analysts are combing their employer’s infrastructure for evidence of threat activity using what they call cyber threat hunting, note Robert M. Lee and Rob Lee in the SANS Institute white paper The Who, What, Where, When, Why and How of Effective Threat Hunting.

What is cyber threat hunting?

According to the white paper A Framework for Cyber Threat Hunting from the people at Sqrrl Data (a company that has roots in the US cyber intel community and is heavily invested in advanced cybersecurity and big data), cyber threat hunting is defined as:

“The process of proactively and iteratively searching through networks to detect and isolate advanced threats that evade existing security solutions.”

The Sqrrl Data white paper continues, “Hunting consists of manual or machine-assisted techniques, as opposed to relying only on automated systems like SIEMs (Security Information and Event Management). Alerting is important, but it cannot be the only focus of a detection program.”

The SANS Institute authors expand on the cyber threat hunting process, calling it an active defense strategy consisting of:

  • Intelligence: The process of collecting data, turning the data into usable information, analyzing the potentially competing sources of that information to produce tactical defense strategy.
  • Offense: The countermeasures organizations may take to defend against cyberattacks, in particular Advanced Persistent Threats (APT).

SEE: Photos: Top 10 cybersecurity issues to watch in 2016

How to hunt for cyberthreats

When cyber hunting, it’s paramount to know where an organization is vulnerable, the best location to go hunting, and how the adversary might try to avoid detection. For example, identify the most valuable assets and data on the organization’s network, so they can be given highest priority defensively, and the hunters can hypothesize how an adversary might try to compromise the assets.

The SANS Institute authors offer the following advice on how to proceed.

What to search

The more data the better. “Hunters need data that will allow them to pivot individual pieces of data into links and correlations that will ultimately reveal the threat,” explains the SANS Institute white paper. “No amount of skilled personnel or expensive tools can make up for a lack of data gathered from the environment, such as flow records, logs, alerts, system events, digital images, memory dumps, and other information gathered from throughout the entire organization.”

How to search

Because of the amount of data, and the need for fast, thorough analysis, use of data science is crucial. To accomplish this the SANS authors mention, “IOCs (Indicators of Compromise), alerts, and other information are useful, but the most effective hunters have access to machine learning and analytics tools, with visual displays to sort this information and help answer their questions and pinpoint abnormal behaviors across large data sets.”

How to focus

Experience has shown that analysis is most effective when it is possible to look past single alerts and identify overall patterns and abnormalities. “Tailored analytics and machine learning make this possible, and automation helps,” suggest the authors of the SANS Institute white paper. “Features in tools such as visual link analysis can help analysts identify an adversary’s larger effort inside the organization even against a backdrop of network noise and with large data sets to filter through.”

How much to automate

The SANS Institute authors warn that cyber threat hunting cannot be completely automated. What can be automated are repeatable processes such as searching for known signs of threats. “But there will always be a need for analysts who have instincts and inquisitive minds,” they caution.

SEE: Software automation policy guidelines (Tech Pro Research)

The human element

What was especially interesting about the two white papers is that the authors of both papers are adamant there needs to be humans involved. Machine learning and big data are subservient and are there to provide the needed information in a way that we humans can understand and use to make good decisions. The SANS Institute white paper, says it best, “What is powerful about threat hunting is that it pits human defenders against human adversaries. The key is to find the right analysts and empower them.”