Patrick Lambert looks at a recent report that tested antivirus and found detection rates "abysmal." But not everyone thinks the tests were fair.
The debate on whether or not an antivirus solution is worth the money spent is not new. There have been surveys and studies comparing the effectiveness of the various security solutions out there for many years. The problem used to be fairly huge, because the very design of an antivirus meant that it would scan a system for potential malware it knows about, and nothing else. In the early years of these security systems, each antivirus would keep a database of known threats, and whenever a new type of malware came in, nothing could detect it, and it would then infect every system it could reach until the companies could update their virus definition databases. Now, this is less so, because of something called heuristics, where an antivirus software not only looks at malware signatures, but also behavior, and tries to detect new malware simply by what the binary file may be doing to your computer. However, the effectiveness of these new solutions is up for debate, and according to a recent study by the firm Imperva, also published in the New York Times, antivirus solutions simply do not do a good job at it.
In its Hacker Intelligence Initiative Monthly Trend Report, published in late December, the researchers picked 82 randomly selected malware files and used them against some of the most popular antivirus solutions to see what their detection rates were. These were newly created infections, taken from web forums, and the result was abysmal, according to the report. The initial detection rate for new viruses was less than 5%. In fact, they found that for some of them, it would take weeks for an antivirus to start detecting the infected file. They also found that the commercial and free solutions had similar detection rates, and recommend that people and businesses stick to freeware products instead. One of the figures they cited, was that 4.5 billion dollars is spent on antivirus solutions — an amount that is not proportional to the effectiveness of these applications. They finally recommend that security teams focus on identifying aberrant behavior rather than detecting infections.
There are many ways to compare security solutions, and it can be very complex to reach a good conclusion. In the weeks following the release of this study, many independent labs and antivirus companies criticized the way this particular research was done. First, the firm used a tool called Virus Total. This site is a very popular one in the security community, where you can upload a file and run it through a series of popular antivirus engines to see if the file is infected. Virus Total gives you a report as to which solutions detected which malware, if any. However, this automated process only uses the core engine of each antivirus solution. It does not use some of the perimeter detection systems and the heuristics will not be as good. It also uses the command line version of the engine, and will not behave like a fully-installed antivirus.
Another problem that the security companies are quick to point out is the small sample size. There are around 142,000 new malicious files being submitted to security researchers every single day. A sample size of 82 is much too small, and could be biased. This is especially true if all of those files were taken from specific Russian forums, for example, and not from a more representative sample of what everyday Internet users may find. Finally, they also note that normal computer users will not face sophisticated threats like Flame or Stuxnet, and that for average malware, your antivirus solution will stop around 9 infections out of 10.
One interesting thing to note is that few people criticised the study for reaching the conclusion that free antivirus solutions were just as good as paid ones. In fact while there are differences between each company, and the features that each antivirus provides, as far as the engine goes, the detection rate is fairly similar, which makes the purchase of a paid software fairly dubious. One conclusion that the study did point out is that some of the free solutions have a higher false positive rate, but this may be seen as a good thing, since it means they might be more aggressive in their detection.
But at the end of the day, the information most of us want to know is whether an antivirus solution is useful, and everyone pretty much agrees on that one. There is no question that using security software is a good thing, and that your antivirus will help detect infections. While the detection rate will never be 100%, modern software with heuristics have a very good rate for normal, average malware, the type we find in abundance on the web. Problems typically occur for new, zero-day malicious infections, and for targeted attacks. This is where researchers don't agree, but it can be safe to assume that if someone is out to get you, then there is a good chance that your antivirus will not protect you. Flame, for example, infected Windows computers in the Middle-East for over four years before antivirus companies finally started detecting it. This was a major failure of the security community, but it was also a new type of highly sophisticated malware.
If you have no reason to think the government or organized crime will spend the necessary resources to break into your system, there probably is no good reason to lose sleep over it. But any modern business should be doing more than simply installing antivirus software. This is just one part of a full protection policy, which should also include intrusion detection systems, log auditing, and a myriad of other things.
What is your take on antivirus? Do you use freeware only, if any? Are there any solutions that you think rise above the crowd?