Firewall Challenge (some results)

Source

History and introduction
Firewall Challenge is a project that replaces our older project Window Personal Firewall Analysis and its subproject Leak-testing. As a part of Window Personal Firewall Analysis project we have deeply analysed security products but we found out soon that such a testing was extremely time consuming. It was not possible to test as many products as we wanted to. On the other hand, Leak-testing seemed to be a very easy way how to test many products in reasonable time. However, Leak-testing is not able to cover many of the important features of the desktop security products. We have decided to combine the simplicity and effectivity of Leak-testing with the scope of our deeper analyses and created this project: Firewall Challenge.
2008-03-25: Results for these products and versions were published:

• Kaspersky Internet Security 7.0.1.325
• Windows Live OneCare 2.0.2500.22

Windows Live OneCare failed the tests completely. This product was probably intended to replace the internal Windows firewall but its protection, as a personal firewall, is nearly the same.

Kaspersky Internet Security confirmed its qualities. Its result is Very good and we can expect that future versions will score even better.

2008-04-25: Results for these products and versions were published:

• Avira Premium Security Suite 8.1.00.206
• BitDefender Internet Security 2008 11.0.16
• BullGuard Internet Security 8.0.0.13
• McAfee Internet Security Suite 2008 9.1.108
• Steganos Internet Security 2008 7.5.509

All these products scored very badly. Other products tested today are anti-virus solutions with very weak personal firewall components.

2008-05-17: Results for these products and versions were published:

• Comodo Firewall Pro 3.0.22.349
• F-Secure Internet Security 2008 8.00.101
• Online Armor Personal Firewall 2.1.0.131 Free
• Panda Internet Security 2008 12.01.00

After the implementation of several new tests, we have decided to retest Comodo Firewall Pro and Online Armor Personal Firewall Free to see if they can still score 100% in our challenge. Both these products lost the perfect score, but Comodo Firewall Pro remained on the first position in our challenge. The odd thing is that the new version of Comodo Firewall Pro did not pass two tests, namely SSS and SSS4, that its previously tested version passed. This firewall has no problems with our new keylogger tests but failed SockSnif test. It also lost a few points in performance tests but its results are still excellent and almost perfect, 95% in total. Online Armor Personal Firewall Free lost more points, especially in keylogger tests. The classification of its protection is now Very good with 89%, close to the Excellent protection, which starts on 90%.
F-Secure Internet Security 2008, Panda Internet Security 2008 and Trend Micro Internet Security 2008 are security suits with very high hardware requirements but their protection is noticeably worse than of other products tested today.

2008-08-01: New results have been published for:

• Dynamic Security Agent 2.0.11.22
• PC Tools Firewall Plus 4.0.0.40
• Sunbelt Personal Firewall 4.6.1839.0

Dynamic Security Agent (DSA) and PC Tools Firewall Plus are both free products and both reached the level 7. DSA scored 62%, PC Tools Firewall Plus 74%. DSA was tested for the first time, PC Tools Firewall Plus was already tested in the past – its previously tested version scored only 6%, so its improvements are remarkable.

Sunbelt Personal Firewall scored worse than before because of the new tests. It did not even reach the level 2 and finished with 7% score.

2008-08-29: New results have been published for:

• Comodo Firewall Pro 2.4.18.184

We have decided to test these two products after we received many requests to test them from our visitors. Users of Windows 2000 asked us to test the old version Comodo Firewall Pro because the new version 3 is not compatible with Windows 2000. Comodo Firewall Pro version 2 scored 55%, which is marked as Poor level of protection, but it is not a bad result if we consider that this version is quite old. Yet still we suggest to switch to another product if you are a user of Windows 2000 or to upgrade to a newer version of Windows and use Comodo Firewall Pro version 3.

2008-09-15: A single product update:

• Norton Internet Security 2009 16.0.0.125

The new version of Norton Internet Security is out. Its improvements are noticeable. While the previous version scored Very Poor 32% (against 62 tests), the new version comes with Good protection level and final score of 71% (against 73 tests).

2008-09-11: A single product update:

• Kaspersky Internet Security 2009 8.0.0.454

We have received many requests to test the new version of KIS. Finally, we have the results. Its previous version tested against 62 tests scored 85%. The new version scored 87% against 73 tests, which is slightly better result, and it is still a Very good protection level.

2008-09-28: New results have been published for:

• BitDefender Internet Security 2009 12.0.10.2
• McAfee Internet Security 2009 10.0.209
• ZoneAlarm Pro 2009 8.0.020.000

Among the 2009 solutions we have tested today, only ZoneAlarm Pro is worth mentioning. Its previously tested version 7 scored only 63% (against 62 tests). The new version has been improved so that it passed all the levels with the final score of 86%, which is a Very good result.

BitDefender Internet Security 2009 and McAfee Internet Security 2009 failed in the tests completely. BitDefender 2008 scored 4% (against 70 tests) and its 2009 version scored 7% (against 73 tests) remaining on None Protection level. McAfee version 2009 finished with the final score of 12% (against 73 tests), its previously tested version scored 6% (against 70 tests).