There is an old saying that is always worth remembering: “There are three kinds of lies: lies, damned lies, and statistics.”
That’s the thought that sprung to my mind in recent days as I read news stories claiming that OS X was the most vulnerable software of the year (e.g. Hackread, SC Magazine, Techworm, Fudzilla, and countless others…).
The news reports stem from a report produced by CVE Details, a website that keeps count of security vulnerabilities based upon their CVE identifiers. According to CVE Details, more new CVE numbers were assigned to Apple during 2015 than any other company.
Yes, more than Adobe or Microsoft or Google or Oracle…
And topping CVE Details’ list, as the product with the most identified security vulnerabilities of all, is Apple’s OS X operating system.
According to the chart, OS X scored an impressive 384 vulnerabilities, marginally ahead of iOS at 375, and then a host of Adobe products, but clearly in front of Microsoft’s first entry — Internet Explorer — in 7th place with 231 vulnerabilities reported.
In all, the researchers counted 654 publicly disclosed security flaws in Apple products, 83 more than Microsoft which came second in the corporate table at 571 vulnerabilities, and well ahead of Cisco (488), Oracle (479), Adobe (460), Google (323), IBM, (312), and Mozilla (188). But again, these statistics can be misleading.
The fundamental problem with charts like this — or rather the news stories they can generate — is that the assumption that more security advisories equates to greater vulnerability is itself, if you’ll excuse the pun, fundamentally flawed.
Why, you ask? Because CVE Details’ chart is not telling us anything about the severity of the vulnerabilities that were found, what the potential impact was, or whether the flaws were ever exploited in the wild.
This means that if a company is doing a good job of finding less serious security holes, and patching them promptly, it could still end up finding itself high up in a chart of what the media might portray as “most vulnerable” software or technology company most plagued with security issues.
In reality, a single critical security hole in a piece of software (such as a remote code execution flaw) could easily outweigh the importance of one hundred less important vulnerabilities found in another.
Although the geek in many of us loves the idea of making lists, and ranking software and vendors in an attempt to determine who might be the least or most secure, simply counting the number of publicly disclosed vulnerabilities is not a good way to go about it.
And, by the way, I’m not arguing that Apple is somehow perfect or that OS X isn’t troubled with security issues. A quick look back on the Intego Mac Security Blog will find any number of articles about serious vulnerabilities that have been found in Apple’s software for OS X and iOS.
But trying to make a sensible conclusion about which software is “most vulnerable” is going to lead you down a path of lies, damn lies, and statistics…
photo credit: flickr photo shared by Cayusa under a Creative Commons ( BY-NC ) license
