Gartner analyst Neil MacDonald wrote a report in March 2012 about security and big data analytics. MacDonald writes, “Information security is becoming a big data analytics problem, where massive amounts of data will be correlated, analyzed, and mined for meaningful patterns.”

MacDonald goes on to mention that traditional security products are unable to scale to the volume of data being captured, thus missing information related to potential attacks or those already in progress. In his report, MacDonald suggests data analytics specifically designed for information security will become necessary.

Fast-forward to today, and MacDonald’s prediction appears to be spot-on. Nearly every business involved in information security is touting how their big-data analytics can help by providing actionable intelligence on suspicious patterns and threats. However, if one looks at current internet attack predictions, the outcome is as bleak, if not more so, than it was back in 2012.

One possible reason

Rick Hofstede, a Ph.D. researcher at the University of Twente – Centre for Telematics and Information, believes he understands why current security measures are not succeeding. In the paper Unveiling SSHCure 3.0: Flow-based SSH Compromise Detection (PDF), coauthored by Hofstede and Luuk Hendriks, the researchers write:

“Due to the sheer and ever-increasing number of attacks on the internet, Computer Security Incident Response Teams (CSIRT) are overwhelmed with attack reports. For that reason, there is a need for the detection of compromises rather than compromise attempts, since those incidents are the ones that have to be taken care of.”

Put simply, Hofstede and Hendriks feel traditional security systems and data-analytic platforms currently in use are not prepared for the exponential growth of connected devices and the vast amount of data being created (potential compromise attempts).

That, in turn, opens the door for warnings of an attack (real compromise) to be missed or lost.

SEE: IoT hidden security risks: How businesses and telecommuters can protect themselves

Flow-based detection with SSHCure

One area where “real compromise” is of utmost concern is Secure Shell (SSH), a network protocol that provides secure authentication and encrypted data communications between computers connecting over an insecure network such as the internet.

Hofstede and Hendriks, focusing specifically on SSH, came up with a solution to the data overload challenge: SSHCure. The University of Twente press release Internet Attacks, Find that Effective One, mentions, “Hofstede chose a ‘flow based’ approach. He looks at the data flow from a higher level and detects patterns; just like you can recognize advertisement mailings without actually checking the content of the brochures.”

The two authors feel the advantages of SSHCure come from command and control taking place at a central location, and the system’s ability to scale easily. The press release adds, “By zooming in on attacks that lead to a ‘compromise’ and require action, Hofstede further narrows his analysis.”

To accomplish this, Hofstede and Hendriks mention SSHCure’s detection algorithms classify SSH attacks into the following categories:

  • Scan: Attackers perform a horizontal network scan to identify active SSH daemons. This phase features a minuscule number of packets per flow, mostly TCP SYN packets.
  • Brute-force: Attackers perform a brute-force attack by issuing many authentication requests against one or more targets (i.e., SSH daemons), typically done by means of dictionaries. The traffic in this phase consists of high-intensity TCP connections commonly referred to as flat traffic.
  • Compromise: Attackers have gained access to a target host by using correct login credentials. This phase typically features either small flows in case of idle connections, or large flows in case the compromised target is being actively misused.

SSHCure needed some revising

Hofstede in his recent Ph.D. defense mentions that earlier versions of SSHCure were far and away the best flow-based brute-force attack detection tools at the time. That said, SSHCure was not ready for production environments because it was yielding too many false positives.

Hofstede believes false positives were occurring because SSHCure’s detection algorithm assumed that brute-force traffic was flat or where all connections have a similar number of packets, number of bytes, and duration length. However, the researchers discovered that brute-force attacks from a remote site do not always appear to be flat.

Hofstede, in his Ph.D. defense, points out the latest version of SSHCure uses a new compromise-detection algorithm that can distinguish non-flat transmissions from remote sources. “His [Hofstede] method proves to be effective, and diminishes the number of incidents, with detection accuracies up to 100 percent–depending on the actual application and the type of network,” adds the press release. “Future, more powerful routers will be able to perform the detection themselves, without needing extra equipment.”

The researchers note version 3.0 is currently available, but there are stability issues. They also state that version 2.4 is still available.