AV-TEST Product Review and Certification Report – September, October 2012

During September and October 2012, AV-test (www.av-test.org ) evaluated 24 home user security products using their default settings.  They used the most current publicly available versions   for the testing. Generally, they were the 2013 versions.  Products were allowed to update themselves   time query their in-the-cloud services.  The test was done on Windows 7.

The   consumer   internet security products were tested in the areas of protection, repair, and usability.  The highest score possible in each category was 6.0.  So, the best possible total score was 18.0. The test was on Windows 7.   You’ll have to go to the av-test.org web site for the individual scores and to download free one-page reports on each of the companies. 

The top ten finishers were, Bitdefender Internet Security (they lost their only point on Usability), F-Secure Internet Security, BullGuard Internet Security, Kaspersky Internet Security, Symantec: Norton Internet Security, Check Point Zone Alarm Free, G Data Internet Security, Trend Micro Titanium Maximum Security, Avast Free, and AVG Internet Security.  Check Point and Avast had the top free products.

BitDefender isn't as well known to the thronging masses as a number of other AV or Internet Security companies.  They are active in the OEM business, offering SDK and rebranding alternatives for endpoint, network,  and cloud protection. In these situations (of course), you may never realize that BitDefender is under the hood.

 http://www.bitdefender.com/oem/#Overview

Microsoft Security Essentials failed the test with 10.5 while McAfee, Norman, and Panda were on the bubble at 11.0. This is a major black eye for the Microsoft Security Essentials product. They scored only 1.5 out of 6 for Protection.  For Removal, 3.5, and Usability, 5.5.

The top ten products are in the table below.  Click on the table to enlarge it.

 

As with all the other tests performed by reputable test groups, there's not a lot of correlation between the best known products and test results.

In March/April testing, the top three products were from BitDefender, F-Secure, and Kaspersky, respectively. 

 http://kensek.blogspot.com/2012/06/av-test-product-review-and.html

 

About AV-Test.org http://www.av-test.org

The AV-TEST Institute is a leading international and independent service provider in the fields of IT security and anti-virus research.  The aim of the research work carried out by AV-TEST is to directly detect the latest malware, to analyze it using state-of-the-art methods and to inform their customers of the top-quality results obtained.

PC Magazine, AV-Test, AV-Comparatives, Internet Security 2013 Mashup

The grid below is a first cut combining Internet Security 2013 products that have been reviewed by PC Magazine and by CNET.  PC World may be added later.  Also on the grid are the ranks from AV-Comparatives on their Demand Detection Test (March) and Real World Detection (August).  The two Norton products on the grid received Editors’ Choice designations. 

The  test results are currently for Internet Security 2012 products. The reviews are for 2013 products. The grid will be modified as reviews are performed and as the Internet Security 2013 solutions are tested by AV-Comparatives and AV-Test. 

Products on the grid initially:  Norton 360, Norton Internet Security 2013, BitDefender Total Security 2013, Kaspersky Internet Security 2013, Trend Micro Titanium Maximum Security 2013, Trend Micro Titanium Internet Security 2013, Panda Internet Security 2013, AVG Internet Security 2013, and  F-Secure Internet Security 2013

Please comment if you feel this grid is useful. 

Best Channel Products 2011 – Business Solutions Magazine

Business Solutions Magazine (BSM) has published the results of their “Best Channel Products 2011" survey. Fortinet, SonicWall, and AVG Technologies dominated the Network Security category. In the area of Network Security, below are the top products from the survey.

Email Security

• AVG Internet Security Business Edition -Top 2 Categories: Ease Of Integration, Ease Of Upgrade
• Fortinet FortiMail-3000C - Top 2 Categories: Reliability/Durability, Features/Functionality

Endpoint Security

• AVG Internet Security Business Edition - Top 2 Categories: Features/Functionality, Reliability/Durability
• Fortinet FortiClient - Top 2 Categories: Reliability/Durability, Ease Of Integration

Unified Threat Management

• Fortinet FortiGate-80C - Top 2 Categories: Features/Functionality, Reliability/Durability
• SonicWALL TZ Series - Top 2 Categories: Reliability/Durability, Features/Functionality

Web Content Filtering

• Fortinet FortiWeb-1000C -Top 2 Categories: Reliability/Durability, Ease Of Integration
• SonicWALL Content Filtering Service (CFS) - Top 2 Categories: Ease Of Integration, Ease Of Upgrade

Fortinet had a pair of products that were finalists for SC Magazine (US) Reader Trust Awards 2011. These were for Best Enterprise Firewall and for Best Integrated Security UTM.

Other categories in the survey were: Data Collection, ECM, General Computing, Managed Services & Networking, Networking, Physical Security (Access Control & Video Surveillance), Point of Sale & Payment Processing, Storage, and Unified Communications & VoIP.

Methodology for Business Solutions Magazine Best Channel Products 2011


Business Solutions Magazine partnered with Penn State University to conduct the survey. During the web survey, they collected 11,711 votes from 1,490 discrete reseller subscribers. Questions in the survey were:

• Richness of Features/Functionality: Does the product meet your customers' functionality needs? Is it easily configurable? Is it easily customizable (i.e. scalable and flexible)?
• Product Reliability/Durability: Does the product meet your quality needs? Do you frequently need to fix the product? Is it durable enough to withstand regular usage?
• Ease of Integration: Is it easily integrated with other products — new or existing?
• Ease of Upgrade: Is it easily upgradeable to meet your customers' needs and technology changes?
• VAR's Ability to Service: Is the product easy for you to service?

For more details on the methodology http://www.bsminfo.com/article.mvc/Best-Channel-Products-2011-Survey-Methodology-0001

For the complete article on Business Solutions Magazine Best Channel Products 2011 http://bsminfo.com/article.mvc/Best-Channel-Products-Page-2011-0001#Index

AV-Comparatives.org Whole Product Dynamic Test – AV-Test.org Certification Test Combined

The table below combines the top 10 products from AV-Test.org and their Q1 certification test on Windows 7 with AV-Comparatives.org and their Whole Product Dynamic Test (January through June). The top 10 products making the cut were BitDefender Internet Security Suite 2011, F-Secure Internet Security 2011, Norton Internet Security 2011, G Data Internet Security 2011, Panda Internet Security 2011, Kaspersky Internet Security, AVG Internet Security 2011, Sophos Endpoint Security and Control 9.5, and Trend Micro Titanium Internet Security 2011, and Eset Smart Security 4.2.

For AV-Test.org (22 products), scores in Protection, Repair, and Usability were averaged together. The top score achievable for each category was 6.0. The product also had to be certified. For AV-Comparatives.org (17 products), the percent of malware blocked during the January through May timeframe was used.

av-test.org and av-comparatives.org Combined Test Results












About the Results

You probably will never see this table on either web site. It’s interesting that the same three products were in the top three in each test. Kudos to them. If this were a top 12 table, Avast and Avira would have made the cut. Five products failed to achieve certification by AV-test.org.

Go to www.av-test.org to see the 1-page reports for each product and to view the individual scores for Protection, Repair and Usability. The table is sortable.

Go to http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests to play with the interactive table. You can also modify the time period and view results.

PC World, PC Magazine, and About.com Internet Security Suite 2011 Results in One Table

The table below consolidates the summary scores of PC World’s “Battle of the Security Superpowers”, PC Magazine’s "Best Security Suites for 2011”, and about.com’s Best Internet Security suites round-ups into a single table.

This table should be looked at in conjunction with the round-ups by the three publications .The PC Magazine "Best Security Suites for 2011" reviews also contain links to comprehensive malware blocking, malware removal, antispam, antiphishing, and performance (boot time, browsing, file move/copy Windows Installer, zip/unzip) charts. One of the below links for PC World shows a comprehensive table summarizing the scores for all the tests in their round up. Within the about.com article, there are a number of links to reports by www.av-test.org.

The first column is the ranking that PC World gave the internet security products. In almost all cases, PC World was more generous in their scoring than PC Magazine. The largest differential was 1.50 with PC Tools Internet Security Suite 2011. This may be because the reviewer found that PC Tools missed a number of phishing sites, the suite's firewall, its antispam capabilities, and that malware interfered with the installation on one system. PCW = PC World, PCM = PC Magazine. The last three columns are the rankings and scores from about.com. The scores are out of 6. Where shown, the final average table is the average from all three publications. In these cases, a perfect score would be 5.33.

This table doesn't include Kaspersky Pure Total Security which received 4 stars in a February 2 PC Magazine review by Neil Rubenking. http://www.pcmag.com/article2/0,2817,2378961,00.asp



Webroot Internet Security Essentials was ranked 8 by about.com with a score of 4.25

Links to Full Articles

http://kensek.blogspot.com/2010/12/pc-world-battle-of-security-superpowers.html

PCW http://www.pcworld.com/article/21461/battle_of_the_security_superpowers.html

PCW http://www.pcworld.com/zoom?id=214618&page=1&zoomIdx=1

PCM http://www.pcmag.com/category2/0,2806,1639159,00.asp

About.com http://antivirus.about.com/od/antivirussoftwarereviews/tp/Best-Internet-Security-Suites-For-2011.htm?nl=1

PC World Battle of the Security Superpowers and PC Magazine Best Security Suites for 2011 Combined

The table below consolidates the summary scores of PC World’s “Battle of the Security Superpowers” and a portion of PC Magazine’s "Best Security Suites for 2011” round-ups into a single table.

This table should be looked at in conjunction with the round-ups by the two publications. The PC Magazine "Best Security Suites for 2011" reviews also contain links to comprehensive malware blocking, malware removal, antispam, antiphishing, and performance (boot time, browsing, file move/copy Windows Installer, zip/unzip) charts. One of the below links for PC World shows a comprehensive table summarizing the scores for all the tests in their round-up.

The first column is the ranking that PC World gave the internet security products. In almost all cases, PC World was more generous in their scoring than PC Magazine. The largest differential was 1.50 with PC Tools Internet Security Suite 2011. This may be because the reviewer found that PC Tools missed a number of phishing sites, the suite's firewall, its antispam capabilities, and that malware interfered with the installation on one system. PCW = PC World, PCM = PC Magazine.

Both publications gave Norton Internet Security Suite 2011 their top ranking at the time the products were tested. PC Magazine's review of Norton Internet Security 2011 was one of the publication's top 10 reviews for 2010, according to their readers. The link below takes you to the article, the slide show and links to each of the 10 reviews:

http://www.pcmag.com/article2/0,2817,2374759,00.asp












Additional internet security suites tested by PC Magazine during the latter half of 2010 not in the table above include, in no particular order: TrustPort Total Protection 2011, BullGuard Internet Security Suite 10, eScan Internet Security Suite 11, AVG Internet Security 2011, Webroot Internet Security Complete 2011, and Ad-Aware Total Security 1.0. More than one of these suites received 2.5 stars, or less. Internet security suites tested and reviewed by PC Magazine during the first half of 2010 but aren't included in the list above. None of these companies went through an initial public offering (IPO) during 2010.

The PC Magazine reviews also discuss functionality such as firewalls, network management, privacy, parental controls, online back-up, data theft protection, full disk encryption, sandbox, and other features that vary by suite.

PC World Battle of the Security Superpowers and PC Magazine Best Security Suites of 2011

These links take you to the full articles.

PCW http://www.pcworld.com/article/21461/battle_of_the_security_superpowers.html
PCW http://www.pcworld.com/zoom?id=214618&page=1&zoomIdx=1
PCM http://www.pcmag.com/category2/0,2806,1639159,00.asp

Addendum - Information Security Magazine - Best Antimalware Security Products, September 2010

Information Security Magazine surveyed readers in a number of different security categories for their Readers' Choice Awards. http://searchsecurity.techtarget.com/magazineFeature/0,296894,sid14_gci1519600_mem1,00.html

In the Antimalware category, Kaspersky Open Space Security, Cisco Security Agent, and Symantec Endpoint Protection were awarded Gold, Silver, and Bronze, respectively.

http://searchsecurity.techtarget.com/magazineFeature/0,296894,sid14_gci1519605_mem1,00.html

SC Magazine Awards 2011 Finalists Announced

February addendum - go to the following url to view the winners http://kensek.blogspot.com/2011/02/sc-awards-2011-us-announced.html

SC Magazine has announced their SC Awards 2011 finalists (US). There are 20 categories for Reader Trust Awards, 5 for Excellence Awards and 5 for Professional Awards. Winners in the latter two categories are chosen by a panel selected by SC Magazine. The Reader Trust Awards are voted on by a panel of SC Magazine readers.

McAfee, Symantec, and Sophos all have multiple products up for Reader Trust awards. Some of the categories security products are finalists for are interesting. For example, in the below, companies up for awards in Best Anti-Malware Management and Best Endpoint Security could be interchangeable. Sophos’ Sophos Endpoint Security and Data Protection is in both categories.


Best Anti-Malware Management (client-based, typically software only)

• ESET for ESET NOD32 Anti-virus 4
• Kaspersky Lab Americas for Kaspersky Open Space Security
• McAfee for McAfee Endpoint Protection Suite
• Microsoft for Forefront Client Security
• Sophos for Sophos Endpoint Security and Data Protection

Best Endpoint/UTM Security

• Check Point Software Technologies for Check Point UTM-1
• McAfee for McAfee Total Protection for Endpoint, Enterprise Edition
• SonicWALL for SonicWALL E-Class Network Security Appliance (NSA) E8500
• Sophos for Sophos Endpoint Security and Data Protection
• Symantec for Symantec Endpoint Protection 11.0

McAfee was recognized for products in 12 areas within the “Readers Trust” competition, which honors best-in-class security products and services. McAfee was also selected as a finalist for three “Excellence Awards." http://www.mcafee.com/us/about/news/2010/q4/20101111-01.aspx

None of the three companies best known for their free internet security software, AVG Technologies, Avast, and Avira are represented this year. AVG Technologies business AVG Internet Security product was a finalist last year in the Anti-Malware category (US Edition).

SC Magazine SC Awards 2011 - Reader Trust Categories

• Best Anti-Malware Solution Gateway
• Best Anti-Malware Management (client-based, typically software only)
• Best Endpoint Security Solution
• Best Computer Forensics Solution
• Best Encryption Solution
• Best Enterprise Firewall
• Best Identity Management Solution
• Best Integrated Security Solution
• Best Intrusion Detection/Prevention Solution
• Best IPsec/SSL VPN
• Best Managed Security Service
• Best Messaging Security Solution
• Best Mobile Device Security Solution
• Best Multi- and Second-factor Solution
• Best Policy Management Solution
• Best Security Information/Event Management (SIM/SEM) Solution
• Best Security Software Development Solution
• Best Vulnerability Management Solution
• Best Web Application Security Solution
• Best Web Filtering Solution

Winners will be announced on February 15 in San Francisco. This will coincide with when the RSA Conference takes place. For a complete listing of finalists in all categories, go to http://www.scmagazineus.com/2011-sc-awards-us-finalists/section/1908/

NSS Labs Consumer Anti-Malware Product Group Test Report – September 2010

NSS Labs has released their “Consumer Anti-Malware Internet Security Product Group Test Report” Internet security products tested were tested and summary guidance for these products are:

Recommend - F-Secure Internet Security 2010, McAfee Internet Security, Trend Micro Titanium Maximum Security.

Neutral - Kaspersky Internet Security 2011, Norman Security Suite, Panda Internet Security 2011, Norton Internet Security 2010.

Caution - AVG Internet Security 2010, ESET Smart Security 4.

NSS Labs Tests

NSS Labs performed 19 tests across three categories - Malware Protection, Exploit Protection, and Performance Impact. The whole report is worth looking at. There is a mix of 2010 and 2011 releases in the report. Results from this report should only be used in conjunction with looking at test and certifications results from other organizations such as Virus Bulletin, AV-Test.org, AV Comparatives, ICSA Labs, and West Coast Labs. PC Magazine uses a collection of malware they receive from internet security vendors in their testing.

NSS Labs was recently featured in an October Forbes Magazine article – “NSS Labs' Nasdaq for Hackers”. It’s interesting reading. In October, “the firm plans to launch an online platform that will allow researchers and penetration testers to buy and sell hacking exploits on an open marketplace known as Exploit Hub.” http://www.forbes.com/global/2010/1011/ideas-internet-hackers-nasdaq-nss-digital-arms-dealer.html

The NSS Labs report can be downloaded at:

http://nsslabs.com/assets/noreg-reports/NSS%20Labs%20Consumer%20Antimalware%20Group%20Test%20Q3%202010.pdf

NSS Labs was previously a member of the Anti-malware Testing Standards Organization www.amtso.org . For more information on NSS Labs, go to http://nsslabs.com

July 2010 Internet Security Performance Test by AV-Comparatives

Testing organization AV-Comparatives issued a 17 page report: “Performance Test (Suite Products). Impact of Security Suites on System Performance”. This was NOT a test on Internet Security Suite effectiveness. Panda Internet Security 2011 received the top score among the products tested. The complete report can be viewed and/or downloaded at http://www.av-comparatives.org/comparativesreviews/performance-tests

Internet Security Suites Tested

The products tested were: avast! Internet Security 5.0, AVG Internet Security 9.0, BitDefender Internet Security 2010, eScan Internet Security Suite 10, ESET Smart Security 4.2, F-Secure Internet Security 2011, G DATA Internet Security 2011, Kaspersky Internet Security 2011, Kingsoft Internet Security 2010, Norman Security Suite Pro 8, Panda Internet Security 2011, PC Tools Internet Security 2011, Symantec Norton Internet Security 2011

The tasks performed (multiple times) to measure performance were - File copying, Archiving / Unarchiving, Encoding / Transcoding, Installing / Uninstalling applications, Launching applications, Downloading files, Worldbench Testing Suite (XP).

Internet Security Solutions Receiving Advanced Certification

Nine companys’ internet security solutions received AV Comparative’s advanced certification level. Panda Internet Security 2011 received the top overall score. Note above which products were 2010 and which ones were 2011 releases. The product’s overall weighted scores were as follows:

1. Panda - 195
2. Kingsoft - 185
3. Norman - 181
4. Symantec - 179
5. eScan - 179
6. ESET - 177
7. Avast - 176
8. F-Secure - 173
9. AVG - 172

The above is a weighted average summary score. The 17 page report has multiple tables and additional details.

Boot Time Measurements Not Part of Testing

The report goes into great detail as to why boot times were not part of the test. In part, they said, “We did not test boot-times on purpose. Security products need to load on systems at an early stage to provide security from the very beginning – this load has some impact on the time needed for a system to start up. Measuring boot times accurately is challenging. The most significant issue is to define exactly when the system is fully started, as many operating environments may continue to perform start-up activities for some time after the system appears responsive to the user.” It’s worth reading the complete explanation on page 4 of the reports, as some vendors “game” their install, to make it look quicker, at the cost of protecting the customer.

Considerations in Selecting an Internet Security Solution

In the end, more critical factors in selecting an internet security solution include: are: which product provides the protection I need, which product provides any additional functionality I need, what is the price. Some of these performance criteria can be used as a tie breaker. They should not be the first cut in selecting an internet security solution. For information on some of the 2011 products that were included in this test, go to. “Internet Security 2011 – The Rush to Release” http://kensek.blogspot.com/2010/08/internet-security-2011-rush-to-release.htm and ‘Internet Security 2011 – The Rush to Release Part II” http://www.mozilla.com/en-US/firefox/central/

About AV-Comparatives

AV-Comparatives is an Austrian Non-Profit-Organization, which is providing independent Anti-Virus software tests free to the public http://www.av-comparatives.org/about-us