NSS Labs report on the Browser Security Comparison report by Accuvant / Google

Image representing Google as depicted in Crunc...
Image via CrunchBase

On 09/12/2011 security reseller Accuvant published report looking at Internet Explorer, Firefox, and Chrome, where Google Chrome comes out on top, and Mozilla Firefox at the bottom. Accuvant are fairly respected, but the report was funded by Google.

Just because Google paid for the report it does not make it corrupt, but it did leave me with a dirt feeling.

The Accuvant report can be found here, I ask you to give it a good read before you read the NSS labs report.

The Accuvant report states that “Readers should understand that, while Google funded the research for this paper, Accuvant LABS was given a clear directive to provide readers with an objective understanding of relative browser security. The views expressed throughout this document are those of Accuvant LABS, based on our independent data collection.”

The Accuvant reports conclusion was that “The URL blacklisting services offered by all three browsers will stop fewer attacks than will go undetected. Both Google Chrome and Microsoft Internet Explorer implement state-of-the-art antiexploitation technologies, but Mozilla Firefox lags behind without JIT hardening. While both Google Chrome and Microsoft Internet Explorer implement the same set of anti-exploitation technologies, Google Chrome’s plug-in security and sandboxing architectures are implemented in a more thorough and comprehensive manner. Therefore, we believe Google Chrome is the browser that is most secured against attack.”

I respect Accuvant but if Microsoft had paid for a report Google would be screaming about its findings. If the findings had been that Internet Explorer 9 was the most secure and Chrome the least, would Google have gracefully have allowed the findings to come to light?

Due to that report being published NSS Labs where approached by several enterprise clients, who asked for a review of the Google/Accuvant publication and supporting tools and data to give an independent opinion.

NSS analysts have also examined data from on-going NSS Labs browser research to give additional guidance.

The NSS Labs report can be found here, give it a good read, its shorter than the Accuvant report.

The final analysis from NSS labs was as follows.

How vulnerable a modem web browser is to attack is certainly a high profile question. It is a well-documented phenomenon that new software contains more bugs (which may expose vulnerabilities) than software that has been around for a while. It is part of the software development lifecycle. Examining historical data on the number of vulnerabilities discovered during a period of time provides insight into the maturity of the software in question and is a good indicator of how many future vulnerabilities will be discovered.”

“The most frequently exploited vulnerabilities leading to system infection with malware are found in third party software such as Java software and the most popular exploit toolkits being used by criminal organizations (such as Black Hole) primarily target third party software (such as Java). This focus on third party applications is likely due to frequent and semi-automatic updates by browser vendors which shrinks the opportunity-time an attacker has to compromise a target, as well as the cross browser-platform nature of third party applications. In essence, if you think from an attacker’s standpoint, why design an exploit for Chrome 15 and a separate one Firefox 8 and a separate one for Internet Explorer 9 when they will have very short shelf life, when a single Java exploit will do the trick just fine? In addition, exploits that use interpreted languages such as Java are very difficult to defend against. Unless they have a pattern match for a known exploit, network and host intrusion prevention systems as well as anti-virus/endpoint protection products are unable to discern legitimate Java from malicious Java. So as an attacker, there is an added “stealth” benefit to exploiting third party software such as Java.”

Google with holding important malware protection from its SafeBrowsing feed so that its own product has an advantage over Firefox and Safari, is an important precedent and contains echoes of accusations made against the company that it improperly provided preferential search results for its own products over third parties. While Google is entitled to improve its product, the way in which the company approached the break with Firefox should be noted.”

So let’s get this right, Google a company whose motto is “Don’t do evil” has dumbed down its SafeBrowing feed that it gives to Firefox and Safari. From a marketing point of view that’s an inteligent thing to do, it makes your browser look better. But from a white hat perspective that’s pretty EVIL with a capital EVIL, you putting you our there as pushing an open form of share security, and then you are making all the users on Firefox and Safari less safe.

Hmm I think you guys need to go read that motto again, because that sticks!

My browser of choice was and still is IE9.

What do you think?


Web Browser Performance and Malware Protection

I was talking to some friends over the weekend, and one of them was blathering on about me being mad to use Internet Explorer 9 and how I should use FireFox 5 instead. So I agreed to play devil’s advocate, and run some home tests between the five top browsers.

Google Chrome
Internet Explorer

I have not install any add ins to the browsers so they are all as clean and stable.

These browsers are running on a Windows PC, that has 4GB of memory, its CPU is an INTEL Core Quad (Q9400 @ 2.66GHz 2.67 GHz). The OS is Windows 7 Ultimate 32 bit, with Service Pack 1 and the latest patches as of 15/07/2011.

The five candidate Web Browsers that I ran my test on are all the latest builds as of 15/07/2011.

Firefox 5.0.1
Google Chrome 12.0.742.122
Internet Explorer 9.0.8112.16421
Opera 11.50 build 1074
Safari 5.0.5 (7533.21.1)

The two tests I ran are the Dromaeo JavaScript Performance Test Suite  and the Sunspider JavaScript Benchmark.

The Dromaeo JavaScript Performance Test Suite
The Dromaeo JavaScript Performance Test Suite is named after the Dromaeosaurs (fast lizard), these lizards stared in the movie Jurassic Park, where they were incorrectly called Raptors. The test suit is created by John Resig (jresig at mozilla.com).

The tests can be run all at once or individually. They are designed to be “real world” in nature by testing a number of features simultaneously.

The Sunspider JavaScript Benchmark
This is a JavaScript benchmark that tests core JavaScript language only, not the DOM or other browser APIs.

This test is designed to compare different versions of the same browser, and different browsers to each other.

Real World
This test mostly avoids microbenchmarks, and tries to focus on the kinds of actual problems developers solve with JavaScript today, and the problems they may want to tackle in the future as the language gets faster. This includes tests to generate a tagcloud from JSON input, a 3D raytracer, cryptography tests, code decompression, and many more examples. There are a few microbenchmarkish things, but they mostly represent real performance problems that developers have encountered.

This test is balanced between different areas of the language and different types of code. It’s not all math, all string processing, or all timing simple loops. In addition to having tests in many categories, the individual tests were balanced to take similar amounts of time on currently shipping versions of popular browsers.

Statistically Sound
One of the challenges of benchmarking is knowing how much noise you have in your measurements. This benchmark runs each test multiple times and determines an error range (technically, a 95% confidence interval). In addition, in comparison mode it tells you if you have enough data to determine if the difference is statistically significant.

TEST ONE – The Mozilla JavaScript performance test suite

In running this test I chose the “Run All Tests” option. It takes about 30 minutes to complete.

Firefox = 1200.42runs’s
Google Chrome = 1114.12runs/s
Internet Explorer = 776.55runs/s
Opera = 1401.06runs/s
Safari = 845.48runs/s

1st Place: Internet Explorer 9.0.8112.16421
2nd Place: Safari 5.0.5 (7533.21.1)
3rd Place: Google Chrome 12.0.742.122
4th Place: Firefox 5.0.1
5th Place: Opera 11.50 build 1074

Click on the links below to see the full results of the tests I ran.
Firefox 5.0.1
Google Chrome 12.0.742.122
Internet Explorer 9.0.8112.16421
Opera 11.50 build 1074
Safari 5.0.5 (7533.21.1)

TEST TWO –  Sunspider JavaScript Benchmark

Firefox = 287.8ms +/- 0.5%
Google Chrome = 385.1ms +/- 3.6%
Internet Explorer = 244.7ms +/- 0.6%
Opera = 282.4ms +/- 1.7%
Safari = 358.5ms +/- 0.9%

1st Place: Internet Explorer 9.0.8112.16421
2nd Place: Opera 11.50 build 1074
3rd Place: Firefox 5.0.1 
4th Place: Safari 5.0.5 (7533.21.1)
5th Place: Google Chrome 12.0.742.122

Click on the links below to see the full results of the tests I ran.
Firefox 5.0.1
Google Chrome 12.0.742.122
Internet Explorer 9.0.8112.16421
Opera 11.50 build 1074
Safari 5.0.5 (7533.21.1)

OK so that’s two for two, so far I have no reasons to swap browser. But what about malware? Will be safer running one of the other browsers.

The NSS Labs have run their 2nd Quarter reports for 2011 on “Web Browser Group Test Socially-Engineered Malware – Europe Q2 2011“. The report looks at a browsers ability to protect a use from Socially-Engineered Malware.

Socially Engineered Malware is one of the most common security threats on the Internet. European users have found themselves particular targets of malware for the 12 months. Eurostat the EU’s statistics office said that almost one-third of internet users in the European Union were victims of malware infections in 2010 despite the having security software installed.

NSS Labs - Web Browser Group Test Socially-Engineered Malware - Europe Q2 2011

NSS Labs - Web Browser Group Test Socially-Engineered Malware - Europe Q2 2011

The NSS labs report focused on URLs that are a significant threat to EU users and follows the same Live Testing methodology as the global tests conducted in Q1 2009, Q3 2009, Q1 2010 and Q3 2010.

Check out NSS Labs Web Browser Group Test for Socially-Engineered Malware attacks in the second quarter of 2011 for Europe.


So the question is will I starting using an internet browser other than Internet Explorer 9?

I find Internet Explorer 9 to be at least as fast as the other browses, and the java tests I ran pretty much proves that feeling.

Safari feels old and crashes.

Chrome I don’t like, and don’t trust.

Forfox its ok, but the final nail in Firefox’s coffin comes from the mouth of Mozilla man himself Asa Dotzler. In a recent exchange on Mike Kaply’s Blog he showed just how much Mozilla in its current form cares for the Enterprise user. You can read the full text at Mikes Kaply’s Blog.

I have paraphrased the three most offence lines from my point of view.

Mike, you do realize that we get about 2 million Firefox downloads per day from regular user types, right?” he wrote. “Your ‘big numbers’ here are really just a drop in the bucket, fractions of fractions of a percept of our user base.”

Enterprise has never been (and I’ll argue, shouldn’t be) a focus of ours. Until we run out of people who don’t have sysadmins and enterprise deployment teams looking out for them, I can’t imagine why we’d focus at all on the kinds of environments you care so much about.”

Years ago, we didn’t have the resources. Today, I argue, we shouldn’t care even if we do have the resources because of the cost benefit trade. A minute spent making a corporate user happy can better be spent making many regular users happy. I’d much rather Mozilla spending its limited resources looking out for the billions of users that don’t have enterprise support systems already taking care of them

So Mozilla in its current form could not give two hoots for Enterprise sysadmins or even Enterprise users so whys should I care about Firefox.

I like Internet Explorer 9, I find it stable, fast, and it renders all the sites I go to perfectly. So I will stick with Internet Explorer 9, and look forward to Internet Explorer 10!