AV-Comparatives Real World Protection Test – August to November 2015

Kaspersky and BitDefender topped twenty companies in AV-Comparatives Real World Protection Test, August to November 2015. These companies finished in the top two, receiving three stars. Both had only one compromised file. Six other companies received three stars over the test period.  Eight of the twenty companies in the test received two stars. Default settings were used for all products.

Trivia question – which North America based malware company received three stars?

Four companies merit the Hall of Shame award for the period, garnering one or zero stars. From the bottom up, ThreatTrack Vipre, Lavasoft, Quick Heat and BullGuard.   Banished to a timeout corner   for being in triple digits for wrongly blocked files – Mcafee, ThreatTrack, and Lavasoft.

An informative graphic in the report depicts the range of protection over the four-month period for each product. The top products were extremely consistent, which is what you would want in a security solution.  The bottom products, less so. 

Not all results are being provided because there is no charge for the report. It can be downloaded at http://www.av-comparatives.org/dynamic-tests/. You can also learn more about the test methodology in the fourteen-page report.  The products tested ranged from free antivirus to internet security suites.  Kudos to AV-Comparatives for detailing some of the statistics methodologies used in compiling their report. Your eyes won’t glaze over as you read about this.

As always, the top products may not be top in terms of number of “likes” they’ve received on their respective Facebook pages.  In  the denouement, should one give more weight to independent third party testing, or a fan club?

An interesting article to read by Adam Winn at San Francisco based OPSWAT,  (www.opswat.com) Sorry Symantec - Antivirus is Not Dead .  Today’s antivirus/malware protection utilizes more than just pattern files and heuristics.

Hall of Shame and timeout corners are not part of AV-Comparatives’ formal designations. You can learn about the organization at www.av-comparatives.org

The trivia question answer – none.  McAfee and Fortinet received two stars.

AV-Comparatives Mac Security Test and Review – July 2015

Austria-based AV-Comparatives  has released the results of their Mac Security Test and Review, July 2015. This report   evaluates ten products users can license for their Mac systems. Products tested were a combination of free and paid solutions. Overall, nine of the products reviewed received AV-Comparative’s Approved Security Product award. 

Malware Tests

Seven of the ten products scored 100% in the Mac Malware Protection Test. None of the tested products scored lower than 98%.

Many Mac security vendors claim that their products detect Windows malware as well as Mac malware. In the Windows Malware Detection Test, seven of the ten products scored 100%. While Macs cannot be infected by these files, the Macs can distribute them, hence the value of testing with Windows malware.

Mac Review and Usability Test

AV-Comparatives used the following criteria in compiling their 64-page review. The appendix provides a comparative checklist that summarizes protection, features, and support for each product. For the test, evaluators use the following as a guideline:

•    Product version reviewed

•    Operating systems supported

•    Additional features

•    Installation

•    Main window

•    Operating system integration

•    Maintenance

•    Non-administrator access

•    Scanning

•    Settings quarantine and logs

•    Malware and phishing alerts

•    Help

 “Our Mac Security Test and Review document comprises a comprehensive evaluation of the ten products we tested,” said Andreas Clementi. “It’s a valuable document that should help enable users to determine which product is the best for their needs. Mac products are not immune from infection by malware, contrary to the belief held by many individuals.  Users consider performing  their own examination of a few products, where 30-day evaluations are available. We don’t recommend not using a security product!”

A more complete list of antivirus programs for the Mac is available at:

http://www.av-comparatives.org/av-vendors-mac

AV-Comparatives performs af  number of tests  over the course of the year. Reports can be downloaded from the company website at:  http://www.av-comparatives.org/  Their “Real World Protection Test March – June 2015” can be found here. Products from Bitdefender, Kaspersky, and Avira were the top three in this test.  

The Mac Security Test and Review can be found at:

http://www.av-comparatives.org/mac-security-reviews/

About AV-Comparatives

AV-Comparatives is an independent organization offering systematic testing that checks whether security software, such as PC/Mac-based antivirus products and mobile security solutions, lives up to its promises. AV-Comparatives offers freely accessible results to individuals, news organizations and scientific institutions. Certification by AV-Comparatives provides an official seal of approval for software performance that is globally recognized.

AVG Technologies in Play, an Alternative Look at Q3 Financial Results

The San Francisco Giants win the World Series in even numbered years.  Rumors circulate about AVG Technology being an acquisition candidate occurs in odd number years. Okay, even numbered as well. Couple that with  so-so financial results? You may want to sell, as well.  

Even before AVG went public, there were “always” rumors about them being for sale as the right price.  Companies being mentioned usually included Hewlett Packard and Cisco. Earlier this year, AVAST Software, an AVG competitor, signed a binding  agreement with CVC Capital Partners for a major investment in the company. The investment valued Avast at about $1 billion US.

Other than throwing off cash for the investors, AVG has   been something of a disappointment. The plan was to go public in early 2012  at $16 to $19.  Instead, they opened and closed around $13. AVG’s market cap, as of 11/15 is just under $1 B.

From a technology standpoint, AVG's growth has been through purchase rather than developing things in house. In September, 2014 they purchased Location Labs, a provider of security for mobile technology. http://now.avg.com/avg-solidifies-leadership-in-growing-mobile-security-market-with-acquisition-of-location-labs/

AVG  entered the mobile security market by purchasing the Israeli firm DroidSecurity in late 2010 DroidSecurity had both a free and paid prospect).  They   increased their share by quietly giving the product away on certain Huawei mobile phones in India (That  announcement appeared on the web and disappeared quickly.  Huawei was being investigated in the 2012 time frame  by the US congress for potentially posing a security threat).

In product testing (ability to stop malware), AVG has failed to be one of the leaders. In AV-Comparatives October Real World Protection tests, AVG came in 10th out of 22. In the September, "File Detection Test of Malicious Software", AVG received on star,finishing 20th out of 22.

(www.av-comparatives.org)  In the Virus Bulletin (www.virusbtn.com ) RAP (Reactive and Proactive test), they weren’t in the top 20. ( https://www.virusbtn.com/vb100/rap-index.xml)

On to the financials. AVG Technologies has their headquarters in the Netherlands. They have an office in Ireland.   Those interested can find multiple stories on the “Double Irish” or “Double Irish Dutch Sandwich”, a technique to significantly g reduce US taxes.  Just saying! Apple and a number of US companies are being creative in using this technique.

For those focused only on revenue (hello analysts), AVG’s 9 months subscription revenue and SMB revenue (less than 15% of their business), is up for the first 9 months of 2014 versus 2013. Trailing revenue, Consumer and Total Revenue, and US Revenue, all down.

For those focusing more on  the bottom line, net income, consumer income, Net Income, Consumer Income, SMB Income, and Operating Income are all down for the first 9 months of 2014.

For those focusing on cash, Net Cash provided by operations is down 35% for the first nine months of this year. The data below is from their latest Form 6-K, available on AVG's web site. 

One would have thought that the positive vibes and karma emanating from the SF Giants home ball park (ATT Park) would have rubbed off on AVG Technologies, given AVG’s US headquarters near proximity to the park. Not the case, however.  

Palo Alto Networks, Check Point top Products in Gartner Magic Quadrant for Enterprise Network Firewalls - 2014

As is probably no big surprise to those in the industry and those purchasing network security products, Palo Alto Networks (PAN) and Check Point had the top rated products in the 2014 Gartner Magic Quadrant for Enterprise Network Firewalls.  The report came out in April.  These are the only two companies in the Leaders Quadrant, with Palo Alto Networks leading on Completeness of Vision and Check Point for Ability to Execute. Fortinet and Cisco were the closest to the in the Challengers quadrant.   The report, ID:G00258296 is available on the PAN web site for those who register. http://connect.paloaltonetworks.com/gartner-mq-2014

Palo Alto Networks pretty much was the originator of the acronym NGFW or Next Generation Firewall, and PAN and Check Point Software Technologies companies compete for many of the same customers. Last year, PAN   introduced their Wildfire infrastructure, enabling the PAN firewall to detect and stop Advanced Persistent Threats (APTs) This is offered to customers via the public cloud or can be deployed as a private cloud.   Gartner also wrote that PAN    was consistently on most NGFW competitive shortlists.  PANS Advanced Persistent Threat Solution   was not among those recently tested by NSS Labs in their April Breach Detection Study.   

Check Point was cited by Gartner as being the market share leader in firewall installed base. They offer an extensive line of security appliances and were also delivered the industry’s first flexible, extensible security architecture, the Check Point Software Blade Architecture.   Check Point’s Anti-Bot Software Blade detects bot-infected machines, prevents bot damages by blocking bot C&C communications. This isn’t a comprehensive Advanced Persistent Threat Solution, but it helps protect the network.

PAN’s product portfolio isn’t quite as extensive Check Point’s,   they do offer a virtualized firewall platform in addition to the more traditional appliance offering, threat subscriptions for URL filtering, and a management platform.  

Fortinet was rated a Challenger by Gartner. They stated Fortinet was “not often beating Leaders in mainstream enterprise selections based on features and vision, nor causing Leaders to react to Fortinet.”

Cisco was rated a Challenger as well.  Gartner didn’t seem them displacing   PAN nor Check Point on the basis of visions or features.  They saw Cisco winning firewall business through channel “execution and “aggressive discounting”.

Juniper Networks completed the trio of companies in the Challenger quadrant.  McAfee was a leader in the Niche quadrant.

Offerings from F5, Arkoon-Netasq, and AhnLab were the furthest down and to the left in the Magic Quadrant.

Check out the complete report.  For an assessment of all sixteen vendors in the report. Some names you’re familiar with may be missing due to consolidation. Gartner also has some brief information on why virtualized firewall penetration is a less than two percent.  “Security-minded enterprises are also rightly skeptical of running firewalls within a hypervisor that is between the threat and the firewall,” according to Gartner. 

 Regarding the Leaders quadrant from the Gartner Magic Quadrant Endpoint report - “A leading vendor isn't a default choice for every buyer, and clients should not assume that they must buy only from vendors in the Leaders quadrant.  Some clients believe that Leaders are spreading their efforts too thinly and aren't pursuing clients' special needs.”

For more details on the Magic Quadrant and how it is created, read “Magic Quadrants and MarketScopes: How Gartner Evaluates Vendors within a Market”.     Sometimes a leader is not the best solution for a particular customer.  Despite that, you will see many   presentations where the vendor uses being in the Leaders quadrant   as a reason to buy from that particular vendor.  Who would have thought that they would do that? www.gartner.com

Some of NSS Labs reports are available at no charge.  www.nsslabs.com

craig kensek

Fire in FireEye Valuation Gets Doused (slightly) With Release of NSS Breach Study Report – He Said, She Said Begins

 NSS Labs issued their Breach Detection Security Value Map on April 2  Neither FireEye nor AhnLab can be pleased.  In brief, the Value Map  measures security effectiveness on the Y-axis and Total Cost of Ownership (TCO) per protected MbPS on the X-axis.  AhnLab and FireEye finished in the dreaded lower left hand corner with FireEye coming in last in security effectiveness (AhnLab was close).  AhnLab had the highest TCO per Protected MBPS. The other four company’s products were in the upper right hand quadrant (Quadrant 1), Fidelis, Fortinet, Trend Micro, and SourceFire. They were all around 98% to 99% effective in NSS testing.  SourceFire was the winner, overall. 

From NSS, “Quadrant 1 contains those products that are recommended for both security effectiveness/management and value.  These devices provide a very high level of protection, manageability, and value for money.”  This document is publicly available from Fortinet as is a detailed report for their FortiSandbox 200D appliance.

http://www.fortinet.com/resource_center/whitepapers/breach-detection-systems-beyond-hype.html

Key findings mentioned in the press release - “Four of Six Leading Vendors Receive Coveted NSS ‘Recommended Rating’”

• Four of six products tested achieved over 95% in overall security effectiveness:   five of the six also received a 0% false positive rate.  AhnLab was the sixth with a 7% false positive rate.  FireEye had the lowest security effectiveness, around 94.5%. 
• Money Doesn’t Always Buy the Best Security: Total Cost of Ownership per Protected-Mbps ranged from $231 to $468 with the highest priced solution,   Conversely, Sourcefire (Cisco) had the lowest TCO and also received one of the highest security effectiveness ratings.
• All BDS Solutions Performed At or Above Vendor Throughput Claims

https://www.nsslabs.com/news/press-releases/nss-labs-reveals-first-security-value-map%E2%84%A2-breach-detection-systems

NSS Labs did not receive any compensation in return for vendor participation; All testing and research was conducted free of charge.

FireEye Stock Price (FEYE)

FireEye stock has dropped 49% percent from its March high of $97.35 to closing at $47.33 on April 11.  52-week range - $33.30 - $97.35.  It will be interesting now to see how the stock performs.  Q1 results won’t be announced until May 6.  Note -  The stock was at   $61.49 on April 2 when the report was released.  FireEye's  Q1 results won’t be comparable to    last year’s Q1 since revenue from their Mandiant acquisition after January 1 will be included.  The stock is up about 15% since the beginning of the year.  NASDAQ is down about 3% ovr the same period of time.

When you’re the market share leader, finishing low in an impartial test, one defense is to attack the attacker.

  

He Said - FireEye

"We are a vendor that specializes in advanced attack detection, not in detecting known, stale samples,” Gupta, FireEye Vice President of Products said.  "We ran their malware samples in our lab and detected every single one of them." A valid test would have used a zero-day exploit to evaluate the detection capabilities of the appliances or, at a minimum, the testing could have been done in a live, customer environment, Gupta added.

FireEye was quick to reply in a blog “Real World vs. Lab Testing: The FireEye Response to NSS Labs Breach Detection Systems Report” At a high level: 

• Issue #1:  Poor sample selection
• Issue #2:  Differing definitions of advanced malware
• Issue #3:  Poor test methodology.   

FireEye offered several paragraphs of detail for each of the above.  It is worth reading the blog.

http://www.fireeye.com/blog/corporate/2014/04/real-world-vs-lab-testing-the-fireeye-response-to-nss-labs-breach-detection-systems-report.html

“The best way to evaluate FireEye is for an organization to deploy our technology in their own environment and they will understand why we are the market leader in stopping advanced attacks, “said Dave Merkel, CTO in an April 2 Network World article.

She Said – NSS Labs

NSS Labs was also quick to replay in a blog “Don't Shoot the Messenger”

Their response is also good reading as most of the response consists of   a 20-bullet point “FireEye Claim” and “NSS Response” table.

“Not everyone can end up in the top right quadrant of the NSS Labs Security Value Map™ (SVM), so it is not unusual for someone to be unhappy.  It is, however, unusual for someone to behave the way FireEye did in this instance.  Normally we would not respond to such attacks, but there are a number of untruths and misdirection’s in their blog post that we feel we must address”, stated Bob Walder, President, and Chief Research Officer at NSS.  “FireEye’s results were not that bad.  The real issue here is that FireEye now has credible competition in the BDS market place and the data from this NSS test shows it.”

https://www.nsslabs.com/blog/dont-shoot-messenger

How Did This Begin

Three companies were tested last summer by NSS Labs in their initial breach study, AhnLab, FireEye, and Fidelis.  Fidelis made their report publicly available and challenged FireEye to do the same.  AhnLab issued a press release about their results, and in a blog went, “FireEye, hello?”  No press release by FireEye on their results.  Demerits to publications not asking about this!  With respect to the three companies, NSS has a multi-page document letting the firms tested know what they can do with the test results.  One thing they can’t do is start-doing comparisons with other companies, combining charts, et cetera from the reports.  The reports were available for purchase.

And What about NSS Labs’ Reputation?

In “IT Security Survey 2014” by  test group AV-Comparatives (www.av-comparatves.org),   issued in February, NSS Labs came in ninth out of 15 vendors.  Over 5800 users responded to the survey.  

 http://www.av-comparatives.org/security-usage-surveys/   

Timing Means Everything When Stock is Sold

On March 12, insider transactions of FireEye stock at $79.54 included: 

1. Norwest Venture Partners IX, LP sold 2 million shares, grossing $160 million.
2. FireEye CTO Aziz Ashar sold 1.04 million shares, grossing $83 million
3. FireEye CEO Dave DeWalt sold 486 thousand shares grossing $38 million

Insiders can’t sell shares whenever they want.  There are windows near the release of financial results that they can’t do anything.  A more comprehensive list of insider transactions can be viewed at

 http://finance.yahoo.com/q/it?s=FEYE+Insider+Transactions

  

It’s difficult to test security products.  Every environment is unique.  The best way for companies to evaluate products is to bring them in and to look at tests by reliable test groups.  The report by NSS Labs probably means   that FireEye will face more testing in house by potential vendors  rather than just be evaluated separately. 

Twitter - ckensek

Just When You Thought the Target Breach Story Was Over. A Tale of Advanced Persistent Threats (APT), FireEye, and Warnings Ignored

In the previous chapter of this adventure, Target CIO Beth Jacob had taken the hit and was going to resign.  Target was going to implement new processes in protecting their network. Prior to this, Target had gone through a number of phases since the attack began in late November – denial, CEO Gregg Steinhafel is  nowhere to be found, “Houston, we’ve got a problem”, “Let’s give customers a ‘we’re sorry’” discount”, CEO is found (finally, some look at a book on crisis management), transparency, free credit watch software for customers, etc.  The Russian hackers involved in this incident were not even very sophisticated with their coding.

Techtarget’s definition of Advanced Persistent Threat – “An advanced persistent threat (APT) is a network attack in which an unauthorized person gains access to a network and stays there undetected for a long period of time.  The intention of an APT attack is to steal data rather than to cause damage to the network or organization.  APT attacks target organizations in sectors with high-value information, such as national defense, manufacturing, and the financial industry.”

In the Bloomberg story “Missed Alarms and 40 Million Credit Card Numbers.  How Target Blew It”, the author writes about how Target HAD Advanced Persistent Threat appliances from FireEye (an APT company that went public several months ago for a gazillion dollars (Side note – FEYE’s  market cap was $10 B as of February 14, though their stock has dropped a bit less than 20% from its high).

The malware had completed most of the phases of the hacker’s objective. Credit card numbers were being stored on a Target server as they were swiped on store terminals. All that was left was for the numbers to be transmitted the cyber criminals for subsequent sale to other cybercriminals.  In November and early December, the hackers went about installing the SW that would send the customer info out to staging points, (probably a botnet), and then to Russia.  Busted!  Well. Sort of. FireEye appliances sent an alert to Bangalore. They alerted the people in Minnesota and…  Minnesota did nothing!  Then, the transmittal of ultimately 40 million records began (a nagging question – was there a DLP (Data Loss Prevention), installed on the network?  It wasn’t until mid-December when the Department of Justice got involved, that Target really began investigating.

By the way, the option for the FireEye appliance to  automatically delete malware as soon as it was  detected was turned off.  What’s even more ludicrous is that Symantec’s Endpoint Protection software, also identified the malware.  $61 million spent by Target so far. Lawsuits, Abysmal Q4 profit (down almost 50%).

Read the Bloomberg/Business Week article. It’s quite interesting.  http://www.businessweek.com/articles/2014-03-13/target-missed-alarms-in-epic-hack-of-credit-card-data

McAfee this week wrote  that this particular attack  was "Far from 'advanced,' The BlackPOS malware family is an 'off-the-shelf' exploit kit for sale that can easily be modified and redistributed with little programming skill or knowledge of malware functionality.”  If this was the case, this is even more embarrassing for Target and their IT team.  http://www.mercurynews.com/business/ci_25322189/mcafee-report-says-target-cyber-attackers-used-common

Takeaways from this - If your network does not have them.  Look at investing in an APT solution.  Look at investing in a DLP solution. Don’t ignore your security solutions when you get flagged. NSS Labs, Ellen Messmer at Network World, and Lawrence Pingree at Gartner.  www.nsslabs.com , www.networkworld.com , www.gartner.com  have all written about Advanced Persistent Threat vendors. Type “advanced persistent threat” into a Google search and a slew of vendors will show up on the RHS.  

Craig Kensek  - Twitter - ckensek

OPSWAT Market Share Analysis of Antivirus, Public File Sharing and Threat Detection - January 2014

 

San Francisco based software company OPSWAT has released their January 2014 Market Share Analysis of Antivirus, Public File Sharing and Threat Detection.

The top 5 vendors on the OPSWAT report were (Vendor market share) 

1. 23.0% - Microsoft
2. 15.9% - Avast
3. 8.9% - AVG
4. 8.1% - ESET
5. 8.0% - Symantec

One interesting product that differs from the others are the offerings from Malwarebytes.  OPSWAT found that Among Malwarebytes users (4.2% share), more than 93% have another product installed, compared to 24% of users of other products.  This indicates that Malwarebytes Anti-Malware and Malwarebytes Anti-Malware Pro are largely used as supplemental products to add additional security to a protected device.  All devices in this data set have at least one antivirus product installed.

 

Using their GEAR technology, OPSWAT looked at detected threats on the endpoint that had AV installed and found that 4.7% had over 10 perceived threats. “The new section in this report that focuses on perceived threats detected by the installed antivirus software provides interesting new data about how actively antivirus products are protecting users, commented OPSWAT employee Alec Stokes. "We’re excited to dive more into this analysis in future reports.”

Other statistics contained within the report include (and more):

 

• Public file sharing files installed
• Hard drive usage
• Operating systems share

 

OPSWAT’s many reports are available at http://www.opswat.com/about/media/reports

OPSWAT has a number of “paid products” as well as a free app remover utility.

About OPSWAT

OPSWAT is a San Francisco based software company that provides solutions to secure and manage IT infrastructure.  Founded in 2002, OPSWAT delivers solutions that provide manageability of endpoints and networks, and that help organizations protect against zero day attacks by using multiple antivirus engines scanning and file filtering.  OPSWAT’s intuitive applications and comprehensive development kits are deployed by SMB, enterprise, and OEM customers to more than ­million endpoints worldwide.  www.opswat.com