Untangle tests AntiVirus tools in LinuxWorld ‘Fight Club’

August 9th, 2007 Posted by - jesler

On August 8th at LinuxWorld, network gateway vendor Untangle performed an all-out public test of different anti-virus vendors to see how they really compare.
In an antivirus “fight club” conducted in front of an audience at the show, 10 antivirus products were confronted with 25 viruses, many submitted by members of the audience. The goal: to see whether the AV tools would catch ‘em all.

The “winners” in last night’s contest were ClamAV, Kaspersky and Symantec. All three tools caught 100 percent of the viruses they encountered. F-Prot and Sophos caught 94 percent; McAfee caught 89 percent; and GlobalHauri, Fortinet, and SonicWall caught 61 percent.

Why were the results so different from what we usually read in tests conducted by other labs? Untangle has an interesting story about it:

... I was talking to one of the testing labs about testing our appliance (for the little sticker certifications). They were eager to work with us for firewall and VPN testing, yet refused to test any antivirus functionality because we were using “the open source solution.” They would not explain why they refused to test ClamAV, although they did offer that they had tested it and it had done poorly relative to the commercial solutions. Given our testing and customer data had shown the exact opposite, we asked for their test results – to which they repeatedly refused. They also repeatedly refused to provide the test data set so we attempt to verify any results they had shown in their labs. I’m left to assume that the testing labs are biased in their testing, probably because they get their funding from the commercial vendors that pay them for testing. Their customers surely wouldn’t be happy if the testing labs claimed a free and open source solution was better.

What’s so cool about the AntiVirus Fightclub? The test is conducted in a public and transparent manner using a real-world test set of malware. Untangle doesn’t test for 0-day malware and doesn’t compare functionalities. It is a simple test of whether or not each vendor’s virus engine catches viruses that have been in the wild in wide distribution.

“What’s surprising about a test like this is how much difference there is between the antivirus products’ performance,” says Dirk Morris, CTO and co-founder of Untangle. “Some of the products you think will do well don’t, and some of the lesser-known products, like open source tools, end up doing well.”

About Untangle:
Untangle was founded with the vision of untangling the complexities of technology, initially targeting network security and control for SMBs. The company follows Open Source development practices to create better code and make it widely available. The Untangle Gateway Platform, the world’s first commercial-grade open source solution for blocking spam, spyware, viruses, adware and unwanted content on the network, provides a free and better alternative to costly, inflexible proprietary appliances. Untangle’s applications are currently being used in hundreds of businesses in a variety of industries, including financial services, real estate, education, bio-science, and professional services. Untangle is located in San Mateo, California.