Security Reviews, Round-ups, and Relevance: Standards & Improved Reviews

Security vendors, from the 3 A’s (Alwil, Avira, AVG Technologies) to the big three in the US (Trend Micro, McAfee, and Symantec) prefer to have their security products perform well in reviews and round-ups. They provide 3rd party validation for their solutions. Security reviews and round-ups, along with tests from valid testing organizations, allow individuals and companies to make informed decisions. The nature of security risks are changing. The AMTSO recognizes this.

It’s worthwhile for reviewers and purchasers, both business and consumers to spend some time on websites such as the Anti-Malware Testing Standards Organization (www.amtso.org) About 40 security vendors and test organizations are members of this organization.

The group’s charter focuses on (from their home page)

1.Providing a forum for discussions related to the testing of anti-malware and related products.
2.Developing and publicizing objective standards and best practices for testing of anti-malware and related products.
3.Promoting education and awareness of issues related to the testing of anti-malware and related products.
4.Providing tools and resources to aid standards-based testing methodologies.
5.Providing analysis and review of current and future testing of anti-malware and related products.

Participants in AMTSO are not trying to shut down or discourage testing. They are trying to raise the standards of testing. They don’t certify any organization’s test. They encourage AMTSO members and others to publicly reference conformity to the guidelines they’ve been developing. Everyone benefits from this.

The AMTSO has a library of documents related to testing, standards, sampling, statistical validity, etc. It’s worthwhile for even the casual blogger or reviewer to look at some of these for guidelines. In particular, they should look at a 5 pager on “The Fundamental Principles of Testing”, http://www.amtso.org/documents.html . Reviewing documents that discuss sampling/sample sizes would also be valuable. All the documents are available to those who agree to the license terms.