Security

Not all AV tools are created equal: Uproar from AV vendors kicks off round two


SecurityIt appears that round two of Untangle's antivirus "fight club," which started in Verdict of “live” test: Not all antivirus tools are created equal, has begun.

McAfee officials and others are taking issue with the methodology of the test in which various proprietary products were pitted against one another, as well as with the open-source ClamAV at the recently concluded LinuxWorld Expo.

The main contention was that the narrow test of just 35 viruses was simply unfair when compared to the hundreds of thousands of malware out in the wild.

According to eWeek, the test consisted of three sets of viruses:

The first batch was a basic test set from eicar.org that Morris [CTO of Untangle] described in a blog as a universal test set. The second set was the "in-the-wild" test of viruses picked from Morris' mailbox that he had received over the years in mass quantities, and the third group of viruses was submitted by users.

The other contention was that "there clearly was a problem with configuration," wrote David Harley, a security researcher. He also added that:

By the tester's own admission... it looks as though the products were tested pretty much "out of the box" without considering whether the conditions of the test would disadvantage specific default configurations.

In my previous posting on this "Fight Club," some of you folks asked for an independent and full-fledged source of test results. Well, it appears that AV-Test is the organization to look at. Under the maxim of "Independent, qualified, and fast," they bill themselves as the "leading company in the range of testing and analyzing antivirus software."

No test results appears to be published on its Web site -- though thankfully, PC Mag has previously posted the results of a recent 'shootout' dated May 22, 2007, from AV-Test. In it, 29 antimalware products were tested against (check this out) 606,901 sets of malware. Products were tuned to their most aggressive detection options.

Here are the top 10 results. I have nothing against ClamAV, but note that ClamAV does not appear in the top few or even top 10 this time round:

Program # Detected Detection %
WebWasher 605,846 99.83%
AVK 2007 604,255 99.56%
AntiVir 603,408 99.42%
F-Secure 594,333 97.93%
Symantec 593,355 97.77%
Kaspersky 592,606 97.64%
Fortinet 589,028 97.06%
Avast! 584,574 96.32%
AVG 583,541 96.15%
Rising 582,772 96.02%

Before you hop over to view the full results, do indulge me by taking the following poll. As noted by several TechRepublic members in my earlier post, many popular AV tools were not included as an option, for which I am deeply apologetic about, despite using AVG Free edition on my personal laptop.

So as part of my penance - there you have it, 20 of them are here this time round. (Which is the max allowable that I can set!)

Now, there appears to be many complaints regarding Symantec AV products not detecting viruses that were easily detected by many other competing AV products.

This is honestly puzzling given its 97.77% detection rate in this pure detection test by an independent party. Its noted slowness aside, my gut feel could be that many malware actively scan for Symantec's AV scanner and somehow disables and/or neutralizes it.

What is your opinion of Symantec's (anecdotal) poor showing?

--------------------------------------------------------------------------------

Stay on top of the latest tech news

Get this news story and many more by subscribing to our free IT News Digest newsletter, delivered each weekday. Automatically sign up today!

About

Paul Mah is a writer and blogger who lives in Singapore, where he has worked for a number of years in various capacities within the IT industry. Paul enjoys tinkering with tech gadgets, smartphones, and networking devices.

144 comments
support
support

I've had Symantec get zapped by a virus several times in the past. Plus it refuses to play well with ANY other A/V. Now I use Avast & Bitdefender Online. when scanning a customer's zombified Compaq with Bitdefender and having Avast active as well, the virus/trojan would try to run and hide from Bitdefender but then get caught by Avast. That was the first zombie PC that I haven't had to reformat the hard drive in order to nuke the malware.

Absolutely
Absolutely

If the complaints were documented instead of anecdotal we could divide the number of complaints about each brand by their market share. Since they are merely anecdotal, we can only suggest, anecdotally, that the popularity of the product in comparison to competitors has [i]something[/i] to do with the popularity of complaining about that product in comparison to complaining about competitiors. McAfee (which was missing an "A" in your poll, btw) gets a slightly smaller share of anecdotal, blanket amnesty. :p

apotheon
apotheon

Several of those examples of security software are not suited to this kind of test -- they were not intended to scan for the types of malware used in the test set, because they are intended for purposes where that malware is not encountered. You noted that ClamAV didn't rate very highly, and at least part of the reason for that is that it was not intended to detect a significant number of the malware threats used for testing. ClamAV is developed as an email gateway scanner. It is not meant to be a general-purpose gateway malware scanner, a desktop system malware scanner, or a system integrity checker. All it is intended to do is scan email on a mail server. Expecting it to detect all of that malware in the test set is a bit like expecting your Toyota Prius to make the trip to Greenland in under eighteen hours. In case you haven't noticed, a Prius isn't intended to float.

ralphcve
ralphcve

What is wrong with using two AV?

richardstevenhack
richardstevenhack

I have used AVG Free and Avast (also free for home users) in the past with no issues. I recommend them to my clients. I've also started using and recommending AOL's AV which is really Kaspersky 6.0 (stripped down) which is also free for both home and company users (although it must be licensed to the individual user, not company wide.) The problem is that home users do not need as high a detection rate or the manageability issues that corporations need. Symantec is a very poor product, not because of its detection rate, but because of its size and complexity and poor quality control. But many corporations choose it because it has manageability tools - which, ironically, are probably why it doesn't work that well. The ClamAV test was bogus from the start and I said so in various places before the test was even done. While ClamAV has reached 100,000 signatures in its database, that has only been in the last six months. And the amount of signatures is not proof of the protective capability of the system, although the more signatures the better. ClamAV is generally good at detecting email viruses - given that it was developed primarily for acting as a gateway on Linux machines to protect Windows machines. It is also good at responding quickly to new viruses. ClamAV may be adequate for home users and possibly very small companies whose ISPs are already scanning their email and who do not have many other malware vectors in play. But it cannot be suitable for anyone who doesn't meet those criteria. One possible exception might be its integration with Spyware Terminator. Spyware is a major issue these days, probably more so than ordinary viruses. Spyware Terminator is an excellent product, and it integrates ClamAV to assists in virus detection as well as spyware. It is also free to both home and corporate users. I would recommend this combination to home users and small business users. I think based on detection rates and usability that Kaspersky 6.0 is probably the best AV for users who need excellent detection, reasonable resource consumption, and manageability.

dirtylaundry
dirtylaundry

Many of my clients, and myself up until recently, use either McAfee or Symantec's Norton AV and often times when I'm called over for noticeable slowness of a system or awkward behavior, I will always manually configure for Spybot S&D to run once at next boot up and 99% of the time it will detect a malware that is called something along the lines of *Microsftdetectionfirewallantivirus_updatedisabled* and once removed I will then run their Norton Liveupdate or McAfee update and it will be several months out of date tho the customer tells me it updates quite often. Such as Internet Explorer is quite popular and there are many anti MS thinkers, malware makers also target Norton/McAfee as it appears to be the ones most used. Often times I do not even have to run Spybot at boot up for these types of malware to be detected, but some will re-activate if I do not. To be sure you are downloading the correct product of Spybot (since several sites spoof the original) head to download.com and type in Spybot and scroll down past the adverts and download the first one with all the stars in its rating. After you download and install it, please remember to update it first. Then after scanning and Fixing the problems, be sure to click Immunize twice - the first time it will run the bar as it finds those products that your system will Immunize, the next click will actually do it to protect from future infections on those variants. This is to be done weekly: Update, Scan/Fix, Immunize x2. And yes, I click the Donate button once a year and send $30. Such a bargain for an awesome product.

asgr86
asgr86

AVG is the best AV i have evry used, all the qualities of a good av are there, its FAST, DECTECTS and cleans, but... Symantec is totaly opposite to FAST . . . . .

Smarttask
Smarttask

I have seen 3 computers in the last month that have had the full internet security program installed from Norton's and they have been full of viruses and malware but when scanned it did not recognize them or know how to fix them. I downloaded Avast and it cleaned them up in no time.

sysop-dr
sysop-dr

Interesting, I can't choose 2 in your poll. I say this because I have always (Since the early 80's when we first started using av and finding virii) suggested to my clients that one AV is not enough. I make the same suggestion with Spyware products as well and when it comes to firewalls, one at your network edge adn one on every system as well. Two of everything, it's called defence in depth and more people should use it. So unable to put my 2 favs in your poll, we use Mcafee and Bit-Defender with regular visits to the on-line Trend Micro Housecall on all Windows systems, just the mcafee and bitdefender on Linux.

btljooz
btljooz

So WHY is it even here?????....no less at the TOP of the list!!! Don't believe me? See this: http://www.download.com/Webwasher/3000-2144_4-10687914.html?tag=lst-0-1 for yourself. ;) And [u]EXTACTLY [b]WHO[/b][/u] was this [i]Third Party[/i] who [u]only[/u] used [b]THREE[/b] "[i]sets[/i]" of viruses for this "[i]test[/i]"??? EH ?:| As, for ClamAV: It's for Linux! Ask yourself what percentage of computers have Windoze installed on them as compared to Linux...which BTW is a LOT more secure than Windoze ever THOUGHT of being!!! You will get the [b][u]MAIN[/u][/b] answer as to why ClamAV didn't make it to THIS [i]list[/i]. :| yeh, I edited it... SEVERAL times, so what?

sboverie
sboverie

About 4 years ago, I was manually removing spyware from a friend's computer. He did not have any AV installed and I did a search to find an online scanning service. The irony was that the spyware that was still installed monitored the search and I got a pop up ad for Symmantec! I found it ludicrous to trust a company for anti spyware when that company was using said spyware to promote itself.

Country Phil
Country Phil

Another source for independent AV comparisons can be found at www.av-comparatives.org. They DO publish the results of their testing, in great detail.

fbrentwood
fbrentwood

>The other contention was that ???there clearly was a >problem with configuration,??? wrote David Harley, a >security researcher. He also added that: >By the tester???s own admission??? it looks as though >the products were tested pretty much ???out of the >box??? without considering whether the conditions of >the test would disadvantage specific default >configurations. I think the product test should be run "out of the box". The majority of people (non-techs) don't know or don't want to know how to change thier AV default settings. Shouldn't the default setting be to provide the best possible protection from virus without hampering performance too much. Out of my circle of friends and family only two of use have the experience and desire to tweek AV to get what we want. Everyone else calls us. So saying a test is unfair by using "out of the box" settings is in fact the only fair way of doing the test. For example: A general implements a battle plan. Whoops the enemy is ready for it. Lets have a do over so the general can put all his best men and equipment into the fray. Unrealistic. Put your best and brightest(AV softaware) against the enemy in the first place. Then if you get beat you know the out come. I use AVG. Easy to use, high quality, low system drain and Free. Sorry Symantec. I can't afford you every year.

OakParkIT
OakParkIT

I use SAV Corporate Edition exclusively at clients needing a server/client solution. I can't think of a better solution for administration from a central location that is as easy to setup, configure and monitor. No AV product is 100% effective. It's up to the IT professional to evaluate the needs and exposure of the client and put in place a blended solution.

Thedan7
Thedan7

Main issue is Symantec/Norton too slow. But like it was stated maybe because it lets too much malware through that is actually the reason for the slowness but it was slow since day one of the install of the 2007 suite (I uninstall everything from that company). Never had issues with viruses getting through. But for now I went to F-Secure. Used ZA in the past maybe go back to that, but for now F-Secure is working. Has anyone had any luck with using a hardware soultion for virus/malware protection that goes between the modem and the router.

MGP2
MGP2

When did Zone Alarm fall off the face of the earth? I have Zone Alarm Security Suite.

santos__tiago
santos__tiago

In my IT and personal life, I ve found that AV solutions only work well if the person in front of the computer allows it to work. I'm sure that many of you have from time to time disabled AV protection, plus letting AV programs be the last to boot up ( comon problem with symantec wich i use ) and more important as pointed out previously not letting the AV program run its scheduled scan, only opens doors for poor AV solution and failure. So is A, B or C better? Sure that a few work better than others, but at the end of the day, it is you the user wich makes the diference.~ If you visit a shady site wich wants to install an activeX, you are the one responsable for your contamination not your AV. I do understand that people want to instal an AV solution and never think about the problem again. But the reality of our big cruel world is completly different. ~ I use for malware and spyware Spybots and Symantec at home and in two other companies. I use MS OneCare in two other places. All of the companies were i am IT, have as main clients, students with usb pens filled with viruses trojan and more from their school's computers. Most typical scene : AV solution blocks access to pen/work to print, reaction from worker : disable AV protection and print the job. Answer block disable option from AV program. Following this management from companies inform me to quickly remove the password protection from the AV, and deal with the problems otherwise. Final solution : backup images of pc at prestine condition locked away. Whenever there is a breakout just put the workstation back to its image. Of course leave the server av solution on 24/7 and check it everyday, since the network can be at any time infected with at least one virus trojan ......

JCitizen
JCitizen

That is the only reason it is not folding up soon! Every once in a blue moon I will find a computer that actually runs with it installed. Perhaps my experience is skewing my perception of Symantec market success. I guess that is my poorly written "anecdotal" response. :(

JCitizen
JCitizen

I had been reading for some while that there were vulnerablilities with the Symantec and other makers AV suites. This is the first report I have read of an actual problem in this area; although I have found compromised computers before using older versions of NIS. And Symantec expects use to trust it with the Microsoft kernal; NOT!! Let me second your opinion of Spybot S&D; I am beginning to wonder why I would ever want any of the bloated retail antispyware products out there! In my opinion they slow the system down so badly that it could be a vulnerablility in and of itself.

binarypc
binarypc

I've reached the point where I've had to recommend AVG to customers. The product does what it's supposed to without a ton of overhead. I actually found a client PC where Norton is chewing up the processor cycles with all it's add-ons so much that they are having problems with Autocad. I'm able to recommend to solve that problem and for a full range of small business products including SBS, Exchange, SQL and client PC's. And, gee... it just works without killing the systems.

paulmah
paulmah

You might have mistook it with another product. I have never used WebWasher, but perhaps the correct link for it might be: http://www.securecomputing.com/index.cfm?skey=1520 The three set of viruses was used in the test by Untangle. This was covered in my previous post at http://blogs.techrepublic.com.com/tech-news/?p=990 If you clicked on the link to view the full list on PC Magazine's site, you will see ClamAV at the bottom 3rd position with detecting 63.81% of the test set. Note that running on Linux or Windows have absolutely no bearing on the test, since this test is purely directed at detection of viruses in the test set.

kurosh
kurosh

I've been interested in finding out more about this product myself, and I noticed that they are rated by Virus Bulletin: http://www.virusbtn.com/vb100/archive/results?vendor=VE49 (you need a free account to view this; there is also a review by them, but you need a subscription to view it) I guess it's only a matter of time before we see more reviews.

jfrickson
jfrickson

Virus Bulletin (http://www.virusbtn.com/ free registration required) currently has test results for 52 AV products. (ClamAV isn't included, however.) They provide full details about how the tests are run, and show the history of each products' test results going as far back as 1998.

RoyKendrick
RoyKendrick

I was dissatisfied with the performance and then returned to the free version of Zone Alarm. What a difference? I'm very satisfied with the results. I wouldn't mind checking out AVG free if I could remember where to find it. No more Norton for me please.

labtrainer
labtrainer

I have used SAV Corp and Panda Corp in small networks. Neither is a joy to use. Panda requires such a thorough house cleaning of any other Av products, including Panda, that it is cost prohibitive. Yesterday I loaded AVG free over Symantec Internet Security on an XP Home machine. It found a Trojan and cleaned it. I had run a scan with SAV the day before. I think I'll take a closer look at AVG products.

JCitizen
JCitizen

Myself, I like SOHO routers that act like gateways. It takes a huge load off your workstations as they don't need the bloat ware - and you can put a lighter footprint on the desktop/notebook,ect. I won't recommend any one product because there are too many new ones out there that probably work better than mine and feature cheaper services. D-Link, Linksys, Belkin, CheckPoint, almost all the popular companies offer what Cisco calls "integrated services" for gateway routers. Not all of them refer to their products as "gateways" though and you'll have to read between the lines to find the models that have these capabilities. Network speed goes way up because the router handles the scanning through SPI and is more capable of doing it separated from your system units where it can use the onboard CPU for such functions. With all the competition someone should be lowering the prices on these services now; but you'll have to do your homework. Their websites don't make it easy to glean information. Of course if your a Linux fan you could do this with an old desktop configured as a web-server slash router; but I digress to those experts who know how to set that up.

HipposRule
HipposRule

.....Kaspersky for it's AV engine

SkatingZebra
SkatingZebra

I believe that Zone Alarm is more well known for their great software firewall than their AV software, so their product may have been overlooked by the testers.

JCitizen
JCitizen

DeepFreeze seems to be the cure all for learning centers I know. I haven't tried it but "unfreezing" the drive to install needed applications and updates seems like a viable solution for me; providing one remembers to reboot first. That doesn't sound any worse than having to logon as administrator to get things done safely.

apotheon
apotheon

. . . and it's a common problem in many niches of the software industry, including OSes, mail and collaboration servers, and office "productivity" suites.

Dumphrey
Dumphrey

has been a staple in my online arsenal for a long time now. My only complaint is that the pop up box for registry change alerts for tea timer never seems to be sized correctly, and I just have to tab n go and hope it chose correctly...

DownRightTired
DownRightTired

well, it was released in North America in March and the Asia-pacific region in July. Oh well, like you said, maybe next round.

Murphy's_Brother
Murphy's_Brother

I use the paid network version at work and the free version at home and on friends and relatives' computers. As Michael Hayes has already posted, it is at www.grisoft.com

apotheon
apotheon

Linux and BSD Unix systems (as well as many other Unix-based systems) have routing capabilities built in. Once you learn the basics of routing and firewall configuration for a given Unix-like OS, it's usually pretty much a cakewalk setting the thing up as a "gateway". There are also firewall/router-specific versions of these OSes that are even easier to set up for such purposes, with many of the system defaults already exactly where you'd want them. I have my eye on m0n0wall for my next firewall/router system.

deanthomas
deanthomas

As a Zone Alarm Security Suite user I have always felt that everything about this package was first class! Always felt snubbed that my opinion was never echoed in these types of tests, never seeing Z.A. even mentioned... I feel better now knowing my anti-virus protection is by the test winner! Thanks!

JCitizen
JCitizen

I'm trained as an engineer so to me an object that is moving has kinetic energy; if you attempt to slow it down or stop it then the inertia carries it through the opposing force. So I defer to the IT way of thinking that you relate here, because I am sure a lot of companies would like to slow Symantec down.

JCitizen
JCitizen

I just got out of the hospital and I got an extreme case of cabin fever! :(

Dusterman
Dusterman

there J ............ . Maybe your isolation there is starting to take it's toll ! ! . :-) . .

JCitizen
JCitizen

as soon as I straighten out my Paypal account I'll send Patrick another one! :8}

JCitizen
JCitizen

Pardon my French btljooz; I haven't made that mistake for a while, sorry! I hate it with a passion when I mistakenly do a double entry like that! Too bad TR hasn't given us the capability to edit an entire post line out!

btljooz
btljooz

to Allow or DisAllow a Registry change with Tea Timer: To "[b]Remember[/b]" the "[i]Decision[/i]" simply put a check mark in the box on the 'confirmation graphic'. Then: press the "(a)" key to [u]Allow[/u] and the "(d)" key to DisAllow the Registry Change. EDITED: Because of optys...ptyos...[b]TYPOS[/b] :8}

Dumphrey
Dumphrey

I still use Spybot regularly. I swear by this program and install it on every computer I set up for friends/family (except my dad, he's on Linux). From my point of view, Spybot is the best overall choice for malware safety. If I HAD to limit myself to one product, Spybot would be that one. And before you ask, no I do not have Spybot posters all over my wall, and there is definitely not a shrine to Spybot in my back room.

JCitizen
JCitizen

he says he dropped the contract with the guy who was writing the code for that applet; and was trying to implement a work around,but I tried one of them and it didn't repair the problem. Hey, with free ware and so little help in the project I can see his side of the issue. I'm sure that the next version will see a solution to that confusing pop-up dialog box. I've been using it so long I know to just click on the left hand box to "remember this" and also on the left radio button to allow the registry change; opposite for the right button.

JCitizen
JCitizen

Thanks to your very considerate input. I hope I can repay you somehow in the future.

Dumphrey
Dumphrey

and the like are router/gateway/av/multi-purpose devices. Most of them are firewall oriented which implies routing, and the rest is added on. For lower end hardware, Coyote Linux offers a free version of their software for personal use. It will run on a 486 with 32Mb of ram. http://www.coyotelinux.com/products.php?Product=coyote As with any firewall software, you will need 2 supported pci ethernet cards, not always easy, as some older mother boards do not support 2 ethernet devices, and some on-board ethernet devices do not like to play nice with pci add on cards....just from my experience btw. I just looked at the mOnOwall site, and that spare box of yours should run it just fine. As for set up, I would use this as the gateway, and have your web-proxy on a different device in-line with this set up as a transparent proxy. A little more config time for a lot more convenience and security. Not as important in a home environment to not be able to bypass the proxy, but I like to be consistant. If I was you, I would (and have) set up an old box with monowall to provide at least minimal functionality in case my router dies. Its well worth spending a few hours poking a stick at.

JCitizen
JCitizen

The only spare unit I got is a 300Mz K6 with 128Mb of RAM; hopefully, if this is enough, I should start playing with something like this: in case my hardware gateway craters. Do you have to set this up on your web-proxy or can it stand alone? I've been away from Linux way too long; gotta get back.

Dumphrey
Dumphrey

monowall, I looked into it a few months back. I was quite impressed by its functioning, and had no real trouble setting it up (other then the odd net card that is always detected as realtek when its a tulip or some such, a short google and I found the right card type....damn...I should have written it down and taped it to the cards backplate...grrr) Another combo I am found of is Debian Etch with Squidproxy, clamav, and squidguard. Dansguardian is simplier to set up filtering for the squidguard, but its a limited license, and consumes more memory then squidguard. Really, as a simple pass through gateway, or web-accelerator, dans and guard are un-needed, except to allow calm av to scan your data stream. I have used a very restrictive version of this set up along with GPOs to limit call center operators to several required sites, and only those sites. Its amazing what someone can do to a computer at 3 am....needless to say, I got tired of cleaning up the random adware (only 1 box was ever really virus ridden, mostly its all spy-ware and ad-ware. Which is scary enough, since they are all on Win2k (fully patched) with updated AV, and no permissions.

dirtylaundry
dirtylaundry

I'm actually having issues with the AV aspect of ZAPro - random scanning that slows my computer and never ends - have to end it via task manager.

Editor's Picks