Who Will Test the Testers?

The anti-malware industry has been plagued since its earliest days by one poorly designed comparative test after another. Contrast this then with the rather surprising fact that anyone with a computer, Internet access, a (perhaps rather undefined) 'Collection' of malware, and a little interest in the subject can open up shop as a computer anti-virus tester, and thereby claim to be a champion of the user and nemesis of the bloated and increasingly beleaguered anti-malware product vendors. This paper summarizes current initiatives by the Anti-Malware Testing Standards Organization and other groups, but also considers next steps, going beyond objectifying methodology, educational issues and blowing away the fog of misinformation and fallacy, to the next level.