PC Matic Dominates with Perfect Virus Detections on the Venerable VB100

In April 2016, we released PC Matic’s Super Shield 2.0 engine at no additional cost. There were numerous improvements in the new engine.

Super Shield 2.0 Improvements

  • enhanced signature based detections
  • enhanced hash based detections
  • enhanced DLL detections
  • 100% internally developed and implemented eliminating our reliance on Vipre’s black list engine
  • new script based detections which block malicious scripts run through Power Shell, wscript, cscript, MSHTA, and runDLL before they execute.

The potential for our new engine was enormous, but we first wanted to prove it in independent tests. In June 2016, we submitted the new engine to the venerable and well respected Virus Bulletin RAP test. They ran their tests in August 2016, and we received the first results in December 2016.

The Virus Bulletin tests are known for their large sample sizes. In this study, there were 50K viruses and 850K good files. These tests are in stark contrast to the rest of the AV testing agencies whose samples sizes typically measure in the hundreds. In order to test such a high volume of viruses, the Virus Bulletin tests are highly automated, and puts extreme pressure on the AV engines. Critics of the Virus Bulletin correctly argue that throwing 50K viruses is extreme and not real world. This is offset by the large sample sizes which provides a wider view of viruses across geographies, threat types, and threat behaviors.

PC Matic’s 2.5 year journey to perfection

We are proud to achieve the industry’s first perfect score on the Virus Bulletin RAP test. This is historic because perfect virus detection is what the market wants and requires. No one wants an AV that catches 90% of the viruses and lets the other 10% enter. We have a perfect score on what is arguably the industry’s most difficult test. So difficult, that many of the larger AV engines have not participated in years.

Critics point to PC Matic’s false positive rate which for this test totaled 13,753. There were 850,000 samples for an accuracy of 98.3%. PC Matic handles false positives in a unique manner. In the event of a false positive, PC Matic blocks the process from executing, but the file is not removed or quarantined. This gives the user the option to address the false positive in six mouse clicks. In parallel, the false positive is uploaded and analyzed by our research team in 24 hours or less. False positives are how PC Matic keeps its white list up to date, much in the way that false negatives are the way that the rest of the industry keeps its black list up to date.

This Virus Bulletin result shows the enormous potential of the white list – to catch every single virus before it executes. The trade off is a 1.7% chance of a false positive, which is an inconvenience. This test result makes the choice clear.

Test Notes

  • The VB100 is a free voluntary test. Virus Bulletin began the RAP test circa 2009. This is the same time frame that products such as McAfee, Norton, Trend Micro, and Webroot discontinued testing with Virus Bulletin. We are working with Virus Bulletin to include involuntary results in future RAP tests.
  • The RAP test is two separate tests, reactive and proactive. For the proactive, Virus Bulletin, installs the security product and disconnects the internet connection for two weeks. During that two week period, Virus Bulletin collects the viruses, and uses the new samples against the frozen security products. In addition to the large samples sizes, that is why some call the Virus Bulletin RAP test, the industry’s most difficult test.

Below is a table with all of the PC Matic Virus Bulletin RAP scores, as well as the highest, lowest, and average test scores.  You may access each report in its entirety by clicking on the month/year.

PC Matic Historic Virus Bulletin RAP Test Results

Date of TestRAP ScoreRAP AverageRAP HighRAP LowPC Matic Proactive ScoreRAP Proactive Average
April 201495.8087.7095.8066.2092.0078.92
August 201495.9890.8898.2040.8089.6080.55
December 201486.8982.4391.6322.5072.2869.72
April 201592.7983.2394.7914.8487.4976.85
August 201592.5275.5692.5225.1086.1065.72
October 201590.7187.3195.9564.386.4778.56
December 201593.4284.9993.5067.7089.8070.57
April 201687.0777.9087.4031.5087.7063.94
August 201699.9081.0399.9041.9099.9066.60
December 201699.9082.6699.9063.399.9064.35

Stop Responding to Threats.
Prevent Them.

Want to get monthly tips & tricks?

Subscribe to our newsletter to get cybersecurity tips & tricks and stay up to date with the constantly evolving world of cybersecurity.

Related Articles