Consumer Reports Tests of Anti-Virus Products Raises Controversy
Home > News	September 1, 2006 12:10 PM

Consumer Reports magazine recently started a major controversy in the PC security world when they created over 5,000 new viruses to test computer security products.

Industry experts have slammed Consumer Reports for creating these new viruses and have raised objections to the testing methodology in general.

We have read the testing methodology and felt there were two things wrong with this study:

1. Creating new viruses for testing purposes seems to be a dangerous and unnecessary practice. Nobody accuses Consumer Reports of having malevolent intentions, but viruses could be released into the wild by accident, causing damage to outside computers.

2. Basing test results on fabricated viruses is misleading. The testers claim that viruses are the "kind you'd most likely encounter in real life". However, they have no way of knowing this. There is no substitute for real-world conditions.

There are two reliable ways to test the efficacy of computer security products. One is to run the products on a test bed of PCs that have been connected to the internet (unprotected) for a long period (months). This technique ensures testing against real-world conditions, but it is not necessarily very thorough.

The second technique is to manually infect computers based upon statistically accurate historical infection rates. This ensures that new and major threats are represented in the test and is generally far more thorough (it may miss little known threats however).

The second approach is the method of choice at this website, a technique that we have used since the beginning of the spyware scourge in April, 2004.

Shameless self-promotion: AdwareReport was the first website to perform objective tests of anti-spyware tests and we continue to perform exhaustive tests of many computer security products each month.