PC Matic Breaks Virus Detection Record on VB100’s RAP Test

vb100rap590px1
PC Pitstop is proud to announce that PC Matic has achieved the best proactive score ever on Virus Bulletin’s RAP test. PC Matic scored 92.8% and the next best score was 87.8% a full 5% behind PC Matic. This result represents independent third party validation that PC Matic’s white list is superior to the traditional black list and heuristic approaches of other products. To be clear, PC Matic’s 92.8% sets a new record for virus detection on the Virus Bulletin RAP test.

The white list feature in PC Matic was introduced in January 2011. Since its inception, the white list is updated daily through crowd sourcing. When a file is blocked because it is not on the white list, a sample is uploaded to our servers. That sample is analyzed by our research team within 24 hours to determine whether 1) it is a new strain of virus or 2) our white list must be updated. Using this approach over the last 3 years, we believed our white list had reached a critical mass so that it would stand the challenges of Virus Bulletin’s RAP test.

In October 2013, On Access security was introduced into PC Matic’s Super Shield. The default protection mode for PC Matic is On Execute which means that PC Matic checks each file when it executes to verify if it is a virus or a legitimate file. With On Access, PC Matic checks every file the operating system touches independent on whether it will be executed or not. For our users, On Access represents an additional layer of security but it comes at the price of performance. This feature was necessary, however, for Virus Bulletin to run the millions of good and bad files through our white list engine without having to execute each one.

The one minus is that PC Matic also failed the Virus Bulletin false positive test. A false positive is when PC Matic identifies a good file as bad. In our case, Virus Bulletin had about one million good files, and PC Matic incorrectly identified about a thousand. This represents a .1% error rate or 99.9% white list accuracy rate.

99.9% accuracy is rather good, but the false positive rate our customers experience is better.

1. Our white list is over three years old and is representative of the files that our users execute. Virus Bulletin does not share files with us, nor should they, but we believe that some of the files are outdated and no longer being executed by real customers.

2. PC Matic’s customer base is predominantly consumer. Our internal correspondence with Virus Bulletin confirms that some of the missed good files are related to large enterprises, which our users are unlikely to see.

3. When PC Matic blocks a potentially good file, the sample goes to our research team, and it is re-categorized in less than 24 hours. So in the unlikely event of a false positive, it effects one and only one customer.

The security industry including the testing houses such as Virus Bulletin, place a higher value on not blocking good files (false positives) than accurately identifying bad files (false negatives). In the Virus Bulletin methodology, there is zero tolerance for false positives. If a product has one false positive out of a million, then the product fails certification. On the other hand, the exact same product can misidentify hundreds if not thousands of bad files as good with little consequence on ratings and certification.

PC Matic employs a black list for scan and clean, and the white list for real time protection. In the event of a false positive, this architecture minimizes the customer impact. The good process is not allowed to run, but it is not erased from the hard drive. The customer can then allow the process to run by entering advanced mode, or placing the process on a local white list. Additionally, as mentioned above, the process has been uploaded to our servers for analysis within 24 hours. In PC Matic’s architecture, a false positive is a one time inconvenience. Compare that to the consequence of allowing a bad file to run. It is far more inconvenient and stressful to allow any bad file to run.

In conclusion, the Virus Bulletin RAP result is independent proof that PC Matic provides superior protection from modern viruses. We, however, are not completely satisfied with the result and have resubmitted to improve our Reactive and Proactive scores on the RAP test. We believe we can set another record with Virus Bulletin, results due out in October 2014.

Below is a graph of all of PC Matic’s historic Virus Bulletin test results.  By clicking on the graphed plots, you will be directed to each individual report.  The reports are also available via the table below, by clicking on the month/year to access the full report. The table also shows the exact RAP score for each test. 

PC Matic Historic Virus Bulletin RAP Test Results

Date of TestRAP ScoreRAP AverageRAP HighRAP LowPC Matic Proactive ScoreRAP Proactive Average
April 201495.8087.7095.8066.2092.0078.92
August 201495.9890.8898.2040.8089.6080.55
December 201486.8982.4391.6322.5072.2869.72
April 201592.7983.2394.7914.8487.4976.85
August 201592.5275.5692.5225.1086.1065.72
October 201590.7187.3195.9564.386.4778.56
December 201593.4284.9993.5067.7089.8070.57
April 201687.0777.9087.4031.5087.7063.94
August 201699.9081.0399.9041.9099.9066.60
December 201699.9082.6699.9063.399.9064.35
(Visited 1,202 times, 1 visits today)

37 thoughts on “PC Matic Breaks Virus Detection Record on VB100’s RAP Test

  1. I see. Hmm. Interesting. You publish this, but is it me or did you do your best to mute the strike through? Baidu and MSecure's strike through's are pretty prominent. PC Matic's strike through is barely perceivable. I just wish your commercials would stop praying on the public that don't know the whole truth. Pure snake oil. I believe you're putting more in your television ads than actual coding and customer support. The very next sentence on VB100 where Rob gets the quote begins; Sadly…

  2. So the CryptoLocker add is very misleading. CryptoLocker itself was shutdown months ago and the keys are freely available for anyone who was affected and still has locked files from Fireeye here:

    https://www.decryptcryptolocker.com

    Calling a defunct virus out as a modern threat in your add is extremely misleading. Sure there are clones out there, there will always be ransomware, they are not CryptoLocker and their code base is not related at all.

    No one can protect Windows XP. You don’t have the source, you can’t patch it. You have no way to know what zero days are there and if they would bypass your whitelisting technology.

    Yes, you use whitelisting which not only has the ridiculous false positive rate you had on the VB100 test, but also is known to work poorly with home users for very simple reasons. Users who use software not on your list will run into issues unless they add it to the whitelist. They are allowed to do so with the product of course or it wouldn’t be useable at all at home.

    It’s been shown time and time again that when you require users to do these actions they become automatic. Call it user fatigue, click fatigue or whatever you want. Users will get used to just adding things like they got used to clicking through browser security warnings, and nothing stops them from adding a trojan to the list. That just becomes more and more likely the more they have to rely on adding third party applications to the whitelist.

    Your false positives were on products from major software vendors. I can’t imagine how bad it would be on software from smaller vendors. Whitelisting is for the Enterprise customer where the business can dictate what is on a machine and what is not, and users have to rely on their employers security team to handle safely adding executables to the whitelist.

    Stop being so misleading. I know you are trying to get people like my grandparents, parents, and other non savvy users to fall for these adds. Frankly I think you should be sued for some of your obvious false claims under the Truth in Advertising act. I don’t know how it hasn’t happened yet.

    Joe

  3. Rob, I've seen your ad on Fox News. For me, it is effective right up to the point when you say that PC Matic lets one's Windows XP PC run as "good" as when it was new. After that error in English grammar, I just don't process the rest of the ad's message. Oh, with one exception: I hear your subsequent claim that native English speakers provide product support. I hear, and I cringe.

    Could be that techies have no problem with the ad's grammar problem. Are you advertising to techies via Fox News? I don't think so. I think you're spending the Big Bucks in hope of attracting ordinary PC users to your product.

    So, won't you please fix that particular error, so that I can enjoy Bill O'Reilly and Megyn Kelly again? And would you please test the ad copy on real people before running ads? Thanks!

    • @rhiltbrunn:

      The one minus is that PC Matic also failed the Virus Bulletin false positive test. A false positive is when PC Matic identifies a good file as bad. In our case, Virus Bulletin had about one million good files, and PC Matic incorrectly identified about a thousand. This represents a .1% error rate or 99.9% white list accuracy rate.

      99.9% accuracy is rather good, but the false positive rate our customers experience is better.

      1. Our white list is over three years old and is representative of the files that our users execute. Virus Bulletin does not share files with us, nor should they, but we believe that some of the files are outdated and no longer being executed by real customers.

      2. PC Matic’s customer base is predominantly consumer. Our internal correspondence with Virus Bulletin confirms that some of the missed good files are related to large enterprises, which our users are unlikely to see.

      3. When PC Matic blocks a potentially good file, the sample goes to our research team, and it is re-categorized in less than 24 hours. So in the unlikely event of a false positive, it effects one and only one customer.

      The security industry including the testing houses such as Virus Bulletin, place a higher value on not blocking good files (false positives) than accurately identifying bad files (false negatives). In the Virus Bulletin methodology, there is zero tolerance for false positives. If a product has one false positive out of a million, then the product fails certification. On the other hand, the exact same product can misidentify hundreds if not thousands of bad files as good with little consequence on ratings and certification.

      PC Matic employs a black list for scan and clean, and the white list for real time protection. In the event of a false positive, this architecture minimizes the customer impact. The good process is not allowed to run, but it is not erased from the hard drive. The customer can then allow the process to run by entering advanced mode, or placing the process on a local white list. Additionally, as mentioned above, the process has been uploaded to our servers for analysis within 24 hours. In PC Matic’s architecture, a false positive is a one time inconvenience. Compare that to the consequence of allowing a bad file to run. It is far more inconvenient and stressful to allow any bad file to run.

      In conclusion, the Virus Bulletin RAP result is independent proof that PC Matic provides superior protection from modern viruses. We, however, are not completely satisfied with the result and have resubmitted to improve our Reactive and Proactive scores on the RAP test. We believe we can set another record with Virus Bulletin, results due out in October 2014.

  4. I have a problem with SSurfoKeepit. It puts ads on every web page. Lots of ads. It even put 3 ads on your PC Matic web page. Please help get this off my computer.

  5. You Liars..

    Here are the real results from VB100.

    The installation process is pretty fast and simple. The interface is fairly basic, but reasonably usable, with only a minimum of options to worry about. Stability wasn’t perfect, with the product collapsing several times under the pressure of dealing with lots of files, even clean ones, and shutting down protection completely several times during the test process.

    Scanning speeds were very slow on demand, and overheads fairly high on access, with a hefty impact on our set of tasks but not too much use of memory or processor cycles.

    Detection was superb in the RAP sets, almost perfect in the reactive sets, and very impressive indeed in the retrospective component. Sadly, this high detection rate comes at a price, with the whitelisting causing alerts on a large number of samples in our clean sets, including components of software from HP, IBM, Microsoft, ATI, Lenovo, Samsung, Lexmark, SAP, Sony and Oracle, as well as a raft of others from smaller software houses and open-source projects. There were also a handful of misses in the WildList sets, in both modes, meaning there is no VB100 award for PC Pitstop this month, despite an interesting effort.

    • @John:

      John: We hope you take the time to review our full response to the VB100 results and the PC Matic record breaking performance.

      In October 2013, On Access security was introduced into PC Matic’s Super Shield. The default protection mode for PC Matic is On Execute which means that PC Matic checks each file when it executes to verify if it is a virus or a legitimate file. With On Access, PC Matic checks every file the operating system touches independent on whether it will be executed or not. For our users, On Access represents an additional layer of security but it comes at the price of performance. This feature was necessary, however, for Virus Bulletin to run the millions of good and bad files through our white list engine without having to execute each one.

      The one minus is that PC Matic also failed the Virus Bulletin false positive test. A false positive is when PC Matic identifies a good file as bad. In our case, Virus Bulletin had about one million good files, and PC Matic incorrectly identified about a thousand. This represents a .1% error rate or 99.9% white list accuracy rate.

      99.9% accuracy is rather good, but the false positive rate our customers experience is better.

      1. Our white list is over three years old and is representative of the files that our users execute. Virus Bulletin does not share files with us, nor should they, but we believe that some of the files are outdated and no longer being executed by real customers.

      2. PC Matic’s customer base is predominantly consumer. Our internal correspondence with Virus Bulletin confirms that some of the missed good files are related to large enterprises, which our users are unlikely to see.

      3. When PC Matic blocks a potentially good file, the sample goes to our research team, and it is re-categorized in less than 24 hours. So in the unlikely event of a false positive, it effects one and only one customer.

      The security industry including the testing houses such as Virus Bulletin, place a higher value on not blocking good files (false positives) than accurately identifying bad files (false negatives). In the Virus Bulletin methodology, there is zero tolerance for false positives. If a product has one false positive out of a million, then the product fails certification. On the other hand, the exact same product can misidentify hundreds if not thousands of bad files as good with little consequence on ratings and certification.

      PC Matic employs a black list for scan and clean, and the white list for real time protection. In the event of a false positive, this architecture minimizes the customer impact. The good process is not allowed to run, but it is not erased from the hard drive. The customer can then allow the process to run by entering advanced mode, or placing the process on a local white list. Additionally, as mentioned above, the process has been uploaded to our servers for analysis within 24 hours. In PC Matic’s architecture, a false positive is a one time inconvenience. Compare that to the consequence of allowing a bad file to run. It is far more inconvenient and stressful to allow any bad file to run.

      In conclusion, the Virus Bulletin RAP result is independent proof that PC Matic provides superior protection from modern viruses. We, however, are not completely satisfied with the result and have resubmitted to improve our Reactive and Proactive scores on the RAP test. We believe we can set another record with Virus Bulletin, results due out in October 2014.

  6. No mention of a firewall test ratings?? I have ESET with malwarebytes with anti-exploit and so far it’s all good. No mention either of any system hogging by PC MATIC so is it as good as described?? IF I can find the answers to the above I might give it a go

    • @Zeke Krahlin:

      Zeke: PC Matic works with Vista.

      We hope you consider reviews posted by actual PC Matic users at other sites like amazon.com, download.com, pcmatic.com/testimonials.

      • Okay, I just did. 3.9 stars out of 5. 190 customer reviews, 39 gave it just one star, four gave it two, and one gave it three. So I’m not impressed. When I duckduckgo’s “pc matic reviews” I slew of angry users on the first result page, alone. It may run on Vista, but I do not want to go there, now that I’m better informed. I’m totally pleased with my jerry-rigged suite of /free/ protection: MSE, PC Threatfire and Malwarebytes. Rarely do I catch a virus or other bad thing…but when it happens, it’s always nipped in the bud. And I mess around a /lot/ with torrent sites and freeware. I have two laptops, one runs Vista, another 7. Then a lovely android tablet.

          • I just read the article /and/ the additional Amazon reviews, and have to say I am not as impressed as I’d like to be. But your companay is definitely on the right track. However, when the reviews are bad, they are /very/ bad…often resulting in a totally crippled system solely because they installed and ran PC Matic. I have a very secure and safe system by using those three programs I mentioned in an earlier comment. But also with Ccleaner. I am talking about /free/ versions only. PC Matic seems to be a fusion of those four applications…with the severe disadvantage of great harm to a significant percentage of users. You don’t know me, but I’ve been around PC’s and cyberspace for quite a number of years. And have even contributed some applications and uniquely creative entertainment never accomplished before, or since.

            I’ve been part of the computer underground since the early years (mid-80’s), and have assisted many new users on how to facilitate one’s access to cyberspace in order to improve their minds and livelihood. But my calling is on the application of global networks to enlighten people on the urgency of ending homophobia…mostly through tales, essays, emails, newsgroup articles, blogs, discussion boards and web pages. Were it not for my dedication to that particular cause, I could easily become one of your most valued contributors to matters of online security, especially when it comes to guiding the average netizen in the right direction for best results w/o risk.

            Freeware security programs abound that in combination can exceed PC Matic’s level of security and privacy, minus any risk. In fact, I originated the concept of antivirus methodology by proposing the use of three major algorithms originally intended to secure the integrity of file transfers. I was extensively quoted in the now-defunct computer magazine Microcornucopeia, on just this subject. That was back in 1988. I’ve scanned that article years ago, which you can still read here:

            http://gay-bible.org/truetales/micro-cornucopia.htm

            Naturally, that link provides the gateway to all my other good works via cyberactivism…as the domain name is as plain as the nose on your face. The home page gives acess to my WordPress blog, Facebook page, and Twitter account as well. So keep up the good work: more dedication and less snake oil, please. But even at this point of PC Matic’s astounding evolution, I can still direct users to totally free applications that do the same thing on a much better, and safer, scale. After such a scathing critique, it is only fair of me to add:

            PC Pitstop’s newsletter rocks…one of the very best out there!

            Most sincerely (and with a dash of CyberHumor):

            Ezekiel J. Krahlin
            Jehovah’s Queer Witness and longterm hacker advocate (I am the founder of Berkeley Unix User Group born in January Y2K)


            “A government is only as good as its operating system.” – Mighty Mouse Virus

            Security Matters & Anti-Matters (or The Mighty Mouse Virus)
            http://gay-bible.org/write/3_security.htm


            I hope I made your day. We are all brothers in the electronic noosphere.

  7. We use PC Matic at a radio station on most of the PC's. I have only had to white list one program (used for audio streaming), it first disabled the service (the program runs as a service). Changing it back to ON and white listing solved the problem.

  8. You have to give that thing a new name. It sounds like a vegetable slicer. The name gives no clue it is a virus checker. The name is completely unmemorable. How about Viravir which conjures up a potent anti-viral drug.

  9. You have to pay the vendor to be tested so that’s why you will not see all the possible virus software companies.

    • @Hank: It is indeed strange that Norton and McAfee the two American leaders do not participate in the VB100 because it is free. Our internal tests indicate that both McAfee and Symantec are middle of the pack on detection rates. That would put it near Kaspersky in detections.

Leave a Reply

Your email address will not be published. Required fields are marked *