Bit9

When you buy a security product, do you want to know how well it did against malware that was out last year? Or do you want to know how well it protect you from attacks in the future? The answer is obvious.

Well apparently organizations like AV-Test.org think you don't care about malware that will come out tomorrow... or even what is out there today. It may shock you to learn how they have been conducting their testing. They basically pre-load a pile of malware on a PC and stick an antivirus solution against it. Effectiveness is measured by how much malware is found and stopped.

So basically - when malware comes onto the machine through an email... when a known vulnerability is patched... when a user visits a webpage that contains a drive-by... all these attacks mean nothing against the test.

Nor does any malware that is coming out today. Or tomorrow. Or even just a couple of days ago. Because the malware that is used for the testing is an old sample that the AV vendors have every opportunity to write specific signatures for. That doesn't represent the way your PCs when they are actually on the Internet. It's a joke!

Here's an article from The Register that is describing how finally, people are thinking about considering a different testing approach that incorporates additional aspects of desktop security like behavioral HIPS and patching and firewalls. It's about time.

Still, how can you trust the results of a test that can't even tell you something so simple as "how infected does a computer on the Internet get with a given protection scheme?"

If you ask me - this is what is wrong with the endpoint security industry today. Too many people patting themselves on the back for fighting malware, and not attention paid to real-world effectiveness.

What do you think? Please comment...