Security

Check out the results of CNET's security vendor survey


By now, we should all know that federal law enforcement is using keyloggers in surveillance activities, and a court ruled it constitutional -- even without probable cause. What this means to the general public is that the U.S. Constitution provides absolutely no protection against law enforcement eavesdropping on our digital lives. Somehow, the fact that it's a computer means none of the usual rules apply.

Many TechRepublic regulars will also be aware of the fact that CNET News.com has published the results of a survey of 13 security software providers that questions their policies toward law enforcement malware -- specifically spyware, such as keyloggers. The results were varied and interesting.

The questions CNET asked were simple. Paraphrased, they were:

  1. Has any law enforcement or government agency approached your company about putting spyware on your customers' computers without a court order and intentionally failing to detect it?
  2. Would your company, at law enforcement request, help hide such spyware from its customers?
  3. Has your company ever received a court order instructing it to comply with law enforcement wishes in this regard?

Including follow-up questions for clarification in some cases, some of the respondents were asked more questions than the others, but at least those three basics were covered. You should form your own opinions of the answers, of course -- after reading the article about the survey.

I've decided to provide a bar graph for a quick and easy look at how, in my estimation, each of the surveyed security vendors fared in terms of trustworthiness as a provider of security software:

Security Vendor Ratings Graph

In the interests of full disclosure, my rating system for the trustworthiness of each security software vendor in regard to protecting against federal law enforcement malware (aka "fedware") -- from 0 to 10 -- looks like this:

  • Not a known malware maker: 1 point
  • Claims about security software behavior are verifiable: 1 point
  • Did not give an offensive answer to the question about contact by law enforcement: 1 point
  • Gave a meaningful, non-suspicious answer to the question about contact by law enforcement: 1 point
  • Did not give an offensive answer to the question about alerting users to fedware: 1 point
  • Gave a meaningful, non-suspicious answer to the question about alerting users to fedware: 1 point
  • Gave an informative answer to the question about alerting users to fedware: 1 point
  • Gave an answer to the question about alerting users to fedware that inspired confidence and displayed great integrity: 1 point
  • Did not give an offensive answer to the question about a court order: 1 point
  • Gave a meaningful, non-suspicious answer to the question about a court order: 1 point

Keep in mind that, in this list, each point that can be gained for the answer to a given question depends on having already gained a point for the question just preceding it. For instance, you can't get a point for a meaningful, non-suspicious answer if your answer actually offends the sensibilities of a responsible security professional. The exception is that it is possible to miss the point for giving a meaningful answer on any of the three questions without missing the point for avoiding offensive answers, simply by failing to answer the question in a manner that does not appear to be a conscious evasion.

Two of these companies, eEye and Sana, scored a 9 out of 10. Considering that there is no way to verify the claims about the behavior of the software with regard to fedware in any of these cases -- all of these companies use closed source, proprietary software that is actually illegal to reverse-engineer under terms of the DMCA -- it is unfortunately impossible for any of them to score a perfect 10 here.

In addition, there is no test suite of which I'm aware for checking the response of this software to various pieces of federal law enforcement malware. As such, there is one point that all vendors missed. Meanwhile, the one point that every single vendor received was the point for avoiding giving an offensive answer to the question about alerting users to the presence of "fedware" on their computers.

Obviously, the scoring system I used is quite subjective. You may very well come to different conclusions about how some of these vendors should rate, even using the same scoring guidelines I listed above -- but I thought carefully about each question, in relation to each vendor, and provided the most objectively valid answers I could.

The results of this survey, summarized and abstracted in the above bar graph, should be on your mind the next time you consider purchasing security software or renewing a software subscription from one of these vendors.

About

Chad Perrin is an IT consultant, developer, and freelance professional writer. He holds both Microsoft and CompTIA certifications and is a graduate of two IT industry trade schools.

3 comments
PrinceGaz
PrinceGaz

You were rating companies based almost totally on how open the various companies were to C-Net about their discussions with government law-enforcement agencies, assuming that a "no comment" was as bad as "we let the feds do anything and don't tell our customers". You gave Microsoft a '1' by your ratings, yet if you read all the answers they gave together, you will see that they have said they will detect and alert users to all keyloggers and other malware regardless of how their source. Of course they have confidential communications with government agencies that can't be disclosed, but the same is true of all large companies. The "no comment" being in your eyes "guilty as sin" is discriminating against the larger security-companies, many of whom would probably see it as bad from a security point of view to chat about such things to some nobody investigator from C-Net even if it along with TechRepublic and other subsidiaries are a significant online prescence. Now that doesn't mean I'd ever rely on Microsoft's current security-products to keep my computers safe, they have some way to go before anyone sane would consider them; but the likes of McAfee who preferred not to answer the two of the three questions (both of which were about government communications) made them the next lowest rated major player. They did however say in the other question, the one they answered in full, that "Yes. McAfee alerts the user to the presence of any spyware or keystroke logger it detects, regardless of who installed it." Surely that is the most important answer and trumps the other two answers which were asking them about confidential matters? While it was nice you took the time to compile your statistics, they are meaningless and it all smacks of sensationalist-journalism.

Absolutely
Absolutely

PrinceGaz: [i]The "no comment" being in your eyes "guilty as sin" is discriminating against the larger security-companies, many of whom would probably see it as bad from a security point of view to chat about such things to some nobody investigator from C-Net even if it along with TechRepublic and other subsidiaries are a significant online prescence.[/i] [edit: added [i]italics[/i] to indicate quoted text] Would you have accepted the scoring system if questions answered "no comment" were given a "?" instead of a "0"? I think the distinction you're drawing between "no answer" and "bad answer" could be well symbolized with such a scoring system, but I also think somebody, such as yourself, who is reading the survey more than casually, for more than a general impression of brands' attitudes toward customer privacy, will do the research you did, and decide for himself whether to interpret the choice to decline to answer as [u]incriminating[/u]. Since we're not in a court of law, I tend to agree with apotheon that non-disclosure is inherently suspicious, and since the raw data was available to you, I think the charges of "sensationalist journalism" & "discrimination" are excessive. Particularly because the entities interviewed are purportedly vying, ie [u]competing[/u] for our business, I consider it very reasonable to expect some disclosure, and to be suspicious of lacks thereof. [edits: fixed typos, and added explanatory content. Sorry, I'll invoke the spiel-chequer (sp?) before I 'Submit My Comments' in future.]

apotheon
apotheon

"[i]assuming that a 'no comment' was as bad as 'we let the feds do anything and don't tell our customers'.[/i]" Not true. A "no comment" would qualify as a "suspicious" answer. A "we let the feds do anything and don't tell our customers" would qualify as an "offensive" answer. I think you may need to have another look at the criteria I described for evaluating responses. I agree that this is not necessarily a good measure of how trustworthy the various vendors actually [b]are[/b], but it gives a clear view of how suspicious one should be of each of them, in my opinion. It may be that you're more suspicious of some of them than you really need to be if you just take my evaluation as gospel, but I tend to figure that when you're placing your security in the hands of a software vendor one should err on the side of greater caution. You might note that general trend with the verifiability criteria: you can't score a perfect ten without your claims being verifiable, a point on which every single surveyed vendor failed -- even those that gave essentially perfect answers to the survey questions. "[i]You gave Microsoft a '1' by your ratings, yet if you read all the answers they gave together, you will see that they have said they will detect and alert users to all keyloggers and other malware regardless of how their source.[/i]" Sadly, that's why Microsoft ended up with a 1 instead of a 0. I suppose I might have weighted the answers individually to provide a better score based on that one single fact, but I chose to simplify things with a one-for-one relationship between answer characteristics and evaluation score modifiers. It's also worth noting that the context within which that answer was finally dragged out of the Microsoft representatives lends a shadow of suspicion to the veracity of Microsoft's claim on that matter (as does the fact that Microsoft didn't directly answer, but had a contracted PR agency handle it instead). I'll let you in on a secret: I was actually looking for excuses to be lenient with Microsoft. I have something of a reputation in some circles for being hard on Microsoft as relates to the security of the software it produces, and I was hoping I'd come up with an evaluation that would provide a more complete impression of impartiality on my part. I just couldn't find any excuses to be more generous with my evaluation of Microsoft. I figured someone would disagree with me -- in fact, I figured several would disagree -- based on the evaluation I did come up with, but I just couldn't justify giving Microsoft any more points. I may provide a more in-depth explanation of the way I rated the various vendors, now that you've brought up the matter and questioned my evaluation, just so you'll have more information with which to judge my results. "[i]While it was nice you took the time to compile your statistics, they are meaningless and it all smacks of sensationalist-journalism.[/i]" I think you're really jumping to conclusions, and failing to look before you leap, if you come to that assessment of the article based solely on the fact that Microsoft scored the lowest of all the surveyed vendors.