When Being an “A” Company Rates a “D”. AV-Comparatives Releases “File Detection Test of Malicious Software” Report

Not a stellar performance by three firms beginning with A in AV-Comparatives March “File Detection Test of Malicious Software.”  Avast – 20th, missing 2.3% of the samples.  AVG Technologies – 21^st, misses 2.5% of the samples, and AhnLab 22^nd, missing 11% of the samples.  Baidu broke the A’s stranglehold on the bottom by leading all companies with 111 false positives, followed by Avast with 95.

Sixteen products did receive AV-Comparatives’ three star designation, led by Kaspersky, F-Secure, and eScan, respectively.  Avira, another A company, also received three stars. 

AV-Comparatives takes care to point out that for this test, “Although very important, the file detection rate of a product is only one aspect of a complete anti-virus product.  AV-Comparatives also   provides a whole-product dynamic “real-world” protection test, as well as other test reports that cover different aspects/features of the products”.

 

The “Whole Product Dynamic Real World Detection" and “File Detection Test of Malicious Software”  tests   are both   available on the AV-Comparatives web site, www.av-comparatives.org

One interesting thing about the products tested in this report is that nine of the engines under the hood in testing were licensed from two companies, BitDefender and Avira.  Details are available in the report.  BitDefender  has an overall detection rate of 99.5%.

About AV-Comparatives

AV-Comparatives is an independent not-for-profit organization offering systematic testing that checks whether security software, such as PC/Mac-based antivirus products and mobile security solutions, lives up to its promises.  Using one of the largest sample collections worldwide, it creates a real-world environment for truly accurate testing. 

Whole Product Dynamic Real World Protection Test – December 2012

It’ll be a few months before AV-Comparatives has a multi-month compilation for this test.  For the month of December though, Trend Micro topped all vendors with a score of 100%.  Congratulations to them!

The top five for the month were:

1. Trend Micro
2. BitDefender
3. F-Secure
4. G Data
5. Kaspersky

Among the best-known vendors known for their freemium products, Avira topped the field.  For the freemium vendors:

09. Avira  

14. Avast (Wins bragging rights for the Czech Republic for the month)

16. Panda

17. AVG Technologies

19. PC Tools (owned by Symantec)

As always, the Whole Product Real World Protection Test is one test.  Individuals or companies evaluating the ability of products to detect and capture malware should visit www.av-comparatives.org , www.virusbtn.com , and www.av-test.org    to view the results of other tests.  These   organizations   use relatively large samples in testing, which improves the legitimacy of the results.  All are more valid than the number of likes on a Facebook fan page.  There is more to product quality than a number of "likes" on a Facebook fan page.

Webroot isn't fond of most real world tests, commenting in a July blog “Webroot Bulletin Regarding AV-Comparatives Results" that,  "Currently most “Real World” tests rely on automation and AV scanners are only given a single chance at detection before the test system is reverted for the next round of testing.  Unfortunately this testing model doesn’t give WSA a chance to leverage our unique cloud approach as it has a very static view of the files being tested.”  They also stated that, "Webroot continues to drive innovation in our products though this isn’t always met with equal innovation in the testing process."  To read the complete blog, go to, http://blog.webroot.com/2012/07/19/webroot-bulletin-regarding-av-comparatives-results/

About AV-comparatives     www.av-comparatives.org

AV-Comparatives is an Austrian Non-Profit-Organization, which is providing independent Antivirus software tests free to the public.  Go to their website to view all the great comparative reports and surveys they publish.  

AV-Comparatives.org Performance Test (Suite Products) - Impact of Internet Security Suites on System Performance (May 2012)

AV-Comparatives.org has released their Performance Test (Suite Products) -  Impact of Internet Security Suites on System Performance . They make  clear that the results are intended to give an indication of the respective internet security suites’ impact on system performance.  The table below has the top five from the test, Webroot, Avira, Avast, Eset, and Sophos.  Also included are the bottom two in the test, Fortinet and PC Tools. 

 http://www.av-comparatives.org/en/comparativesreviews/performance-tests   

 

The second through fourth columns are all tests by AV-Comparatives.org.  The second column is a subset of the results from their performance test.  Test cases for this involved file copying,  archiving/unarchiving,  encoding/transcoding, installing, uninstalling, and launching applications, and downloading files.

The rankings  from AV-Comparatives'  May  2012 Whole Product Dynamic Test, and overall results from their March On Demand Detection Test are in the next two columns.  BitDefender came in a five-way tie for first in the Whole Product Dynamic test with 99.1% blocked.  G Data took top honors on the On Demand Detection Test.

The last column is from AV-Test.org   and their March/April testing on consumer internet security suites.  They scored the products on Protection, Repair, and Usability.  BitDefender Internet Security Suite was the top product with 17 out of 18 points.  www.av-test.org

As you would expect, performance, as in speed/impact on system performance,  means little when it comes to how well a product stops malware (though it does give a company some bragging rights).   In looking at the other columns, it shows that to  make an informed decision, you should look at multiple tests (not a lot of correlation between the three columns).  However, given the double-digit rankings for the products that were the “fastest” in the other tests, you should look to see which products topped the testing. 

About AV-Comparatives

AV-Comparatives is an Austrian Non-Profit-Organization, which provides independent Anti-Virus software tests free to the public. Go to the AV-comparatives website for complete details about the organization, the many tests they perform, and to download copies of test reports.

www.av-comparatives.org

August through November 2011 AV-comparatives.org Whole Product Dynamic Test

AV-Comparatives has released the Whole Product Dynamic Test for the August through November timeframe. The top five products are listed below. The top five products in the March through June timeframe – Symantec, F-Secure, BitDefender, G Data, and Trend Micro. All of these were in the top 10 in the latter half of the year.

August through November Results AV–comparatives.org Whole Product Dynamic Test

1. 99.5 – Symantec
2. 99.4 – BitDefender
3. 99.1 - Kaspersky
4. 98.9 – Qihoo
5. 98.8 – Trend Micro

Of the above, only Trend Micro did not receive the Three Star Advanced Rating. F-Secure and G Data did, however. PC Tools and Webroot were at the bottom for the latter half of the year. They received the not so coveted “Tested” designation.

To see complete results for the latter half of the year (17 vendors), go to http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests

The burning question for 2011 – How many toothpicks does it take to safely surf Facebook?

About AV-comparatives.org

AV-Comparatives is an Austrian Non-Profit-Organization, which is providing independent Anti-Virus software tests free to the public. Free is a good thing.

www.av-comparatives.org

AV-comparatives.org Whole Product Dynamic Test – September 2011

The top five products in the AV-Comparatives.org and the "Whole Product Dynamic Real World Test” for September has some major antivirus and internet security suite vendors performing not so well. The top five for September were BitDefender, F-Secure, Trend Micro, McAfee, and Symantec. Avira and K7 dropped out of the top 5. K7 dropped to fourth from the bottom, while Sophos, WebRoot, PC-Tools cannot be smiling, rounding out the bottom three. Avira was the top “free” vendor. This test may not greatly effect revenues, but there are bragging rights in these tests;).

September Results Whole Product Dynamic Test

1. 99.7 – BitDefender
2. 99.0 – F-Secure
3. 98.8 – Trend Micros
4. 98.7 – McAfee
5. 98.7 - Symantec

August Results Whole Product Dynamic Test

1. 99.5 – Trend Micro
2. 99.0 – Symantec
3. 98.4 – Avira
4. 98.4 – BitDefender
5. 97.9 – K7

To see complete results for September (twelve more vendors), go to http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests and view the monthly report. More important though, are results over a period of time. For a one quarter write-up http://www.av-comparatives.org/images/stories/test/dyn/wpdt2011_1_en.pdf

The 2012 versions of antivirus and internet security suites have been rolling out of the last couple of months. These will probably be incorporated into the test bed, replacing the 2011 versions. This test group continues to innovate in their testing. Go to their website to check out the anti-phishing test (to be covered another time).

Go to http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests to play with the interactive table. You can also modify the time period and view results.

About AV-comparatives.org - www.av-comparatives.org

AV-Comparatives is an Austrian Non-Profit-Organization, which is providing independent Anti-Virus software tests free to the public.

AV-comparatives.org Whole Product Dynamic Test – August 2011

The top five products in the AV-Comparatives.org and the "Whole Product Dynamic Real World Test” for August has some major antivirus and internet security suite vendors performing not so well. The top 5 for the month were Trend Micro, Symantec, Avira, BitDefender, and K7.

1. 99.5 – Trend Micro
2. 99.0 – Symantec
3. 98.4 – Avira
4. 98.4 – BitDefender
5. 97.9 – K7

To see complete results for August (twelve more vendors), go to http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests and view the monthly report. More important though, are results over a period of time. For a one quarter write-up http://www.av-comparatives.org/images/stories/test/dyn/wpdt2011_1_en.pdf K7 has improved a lot in the August results.

To see an older table of the AV-Comparatives.org Whole Product Dynamic Test and AV-Test.org Certification Test Combined, go to http://kensek.blogspot.com/2011/07/av-comparativesorg-whole-product.html . The 2012 versions of antivirus and internet security suites have been rolling out of the last couple of months. These will probably be incorporated into the test bed, replacing the 2011 versions.

Go to http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests to play with the interactive table. You can also modify the time period and view results.

About AV-comparatives.org - www.av-comparatives.org

AV-Comparatives is an Austrian Non-Profit-Organization, which is providing independent Anti-Virus software tests free to the public.

AV-comparatives.org Whole Product Dynamic Test – June through July 2011

A bit of a switch in the top five products in the AV-Comparatives.org and their Whole Product Dynamic "Real World" Test, June through July, versus the January through June test. These results were out of the box capture rates.

1. 99.3 - F-Secure
2. 99.3 - Panda
3. 98.8 - Trend Micro
4. 98.3 - BitDefender
5. 98.1 - G Data

Symantec/Norton dropped to seventh, blocking 98.1%, out of the top five. The top six for the January through June time period were BitDefender, F-Secure, Symantec, G Data, Panda, and Kaspersky.

To see a table of the AV-Comparatives.org Whole Product Dynamic Test and AV-Test.org Certification Test Combined, go to http://kensek.blogspot.com/2011/07/av-comparativesorg-whole-product.html


Go to http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests to play with the interactive table. You can also modify the time period and view results.

About AV-comparatives.org - www.av-comparatives.org

AV-Comparatives is an Austrian Non-Profit-Organization, which is providing independent Anti-Virus software tests free to the public.

AV-Comparatives.org Whole Product Dynamic Test – AV-Test.org Certification Test Combined

The table below combines the top 10 products from AV-Test.org and their Q1 certification test on Windows 7 with AV-Comparatives.org and their Whole Product Dynamic Test (January through June). The top 10 products making the cut were BitDefender Internet Security Suite 2011, F-Secure Internet Security 2011, Norton Internet Security 2011, G Data Internet Security 2011, Panda Internet Security 2011, Kaspersky Internet Security, AVG Internet Security 2011, Sophos Endpoint Security and Control 9.5, and Trend Micro Titanium Internet Security 2011, and Eset Smart Security 4.2.

For AV-Test.org (22 products), scores in Protection, Repair, and Usability were averaged together. The top score achievable for each category was 6.0. The product also had to be certified. For AV-Comparatives.org (17 products), the percent of malware blocked during the January through May timeframe was used.

av-test.org and av-comparatives.org Combined Test Results












About the Results

You probably will never see this table on either web site. It’s interesting that the same three products were in the top three in each test. Kudos to them. If this were a top 12 table, Avast and Avira would have made the cut. Five products failed to achieve certification by AV-test.org.

Go to www.av-test.org to see the 1-page reports for each product and to view the individual scores for Protection, Repair and Usability. The table is sortable.

Go to http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests to play with the interactive table. You can also modify the time period and view results.

AV-Comparatives Releases Preliminary Results For Their 2011 Whole Product Dynamic Test

AV-Comparatives has released their preliminary results through April 2011 of their Whole Product Dynamic Test. The companies with the top three products (preliminary results) are in order – Symantec, BitDefender and Eset. Avast in eighth, was the top free product. Seventeen companies were in the test. http://chart.av-comparatives.org/chart2.php Results will be published in July. The latest revision for this test came out in April.

Notes for how the test was set up are available at http://www.av-comparatives.org/images/stories/test/dyn/wpdt-general.pdf

In the August – November 2010 Whole Product Dynamic Test, F-Secure, Symantec, Avira, and Kaspersky had the top four products and received an advanced “three star" rating for the test. F-Secure fell to sixth and Avira to tenth in the preliminary Whole Product Dynamic Test for 2011.

About AV-Comparatives

AV-Comparatives is an Austrian Non-Profit-Organization. They provide independent Anti-Virus software tests free to the public. www.av-comparatives.org

AV-Comparatives Whole Product Dynamic Test – August through November 2010

AV-Comparatives released their latest Whole Product Dynamic Test results on December 17. Nineteen products are in the report. Four vendors’ consumer internet security solutions received the Advanced, 3 stars rating in the Whole Product Dynamic Test. These were F-Secure, Symantec, Avira, and Kaspersky (Symantec and Kaspersky were the only products to receive 3 stars in the December 2009 report). F-Secure and Symantec showed the least variability of monthly results. These four vendors, and solutions from G DATA and Avast, were the only ones to score above 95% during all test months. A variety of malware was used in the test. Out of the box settings for the products were used.

The only one of these four companies failing to receive 3 stars in AV-Comparative’s Proactive/Retrospective Test (last revised on December 6) was Kaspersky. Twelve products received 3 stars in this test. Consistency can be a good thing. The most notable vendor missing from both of these tests was McAfee.

Eight products received a two star rating in Whole Product Dynamic Test. Their average results were all above 95%.

For the complete report and to look at interactive charts, go to the AV-Comparatives web site.

AV-Comparatives is an Austrian Non-Profit-Organization, which provides independent Anti-Virus software tests free to the public. www.av-test.org

Note that there are other testing organizations as well. Individuals and companies evaluating AV and Internet Security SW should also look at the others, at reviews, round-ups, and group tests performed by reputable testing organizations, and technology publications. For information on testing best practices, people can go to the Anti-Malware Testing Standards Organization website, www.amtso.org

AV-Comparatives Whole Product Dynamic Test – August through October 2010

AV-Comparatives (www.av-comparatives.org) released their Whole Product Dynamic Test report for August through October 2010 on November 22. 14 AV/Internet Security products are in the report, comprising three months of tests. The products were a mix of 2010 and 2011 releases (and for some of the products changed in September or October) For some products, this may have effected the results. For Kingsoft, the results improved slightly. For AVG, the results were worse. For AVG Technologies, the rate was above 95% for August and September before falling to below 90% for October. Kingsoft and Norman were the only products below 90% for October. A variety of malware was used in the test, with out of the box settings for the products being utilized.

The table below should only be looked at in conjunction with the whole report! Updates to this report will be released regularly. It will be great to see any trends that develop or changes in rankings. The report is available at www.av-comparatives.org.

Key to the below – U signifies the percentage was up (improved) between August and October, S signifies the percentage was the same; D signifies that the percentage was down.

Percent Compromised Ranking for October

----Below 100%----

1U - Symantec
2S - F-Secure
3U - GDATA
3U - Avast
5S - Kaspersky
6D - Panda
6D - Avira
8D - PC Tools
9D - ESET

----Below 95%----

10D - Trend Micro
11U - BitDefender
12D - AVG (above 95% in August)

-----Below 90%-----

13U - Norman
14U - Kingsoft

Six products went up between August and October, two were the same, and six went down.