A lot is happening in the browser world this year. Google Chrome recently passed 1 billion users, Mozilla revealed big changes are coming to Firefox, and Microsoft is finally building its successor to Internet Explorer, called Edge. Clearly, the next browser battle is brewing, and I really can’t think of a better time to bring back my benchmark rundown from nearly five years ago.
That said, while Windows 10 is supposed to hit RTM this week, it won’t be available until July 29. Even then, most Windows users will only be able to get it weeks, if not months, later.
[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":1766339,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,","session":"A"}']And hence why this is a “beta browser benchmark.” I’m using the term “beta” loosely to acknowledge that Microsoft Edge isn’t ready for prime time yet, and I figure it’s only fair that we test Firefox Nightly and Chrome Canary rather than their stable counterparts.
A benchmarking program that can test browsers in a definitive way simply does not exist. As such, we’re going to go through eight benchmarks. We’re also only focusing on Windows to maintain a common platform, and since that’s what the larger majority of desktop users browse on.
AI Weekly
The must-read newsletter for AI and Big Data industry written by Khari Johnson, Kyle Wiggers, and Seth Colaner.
Included with VentureBeat Insider and VentureBeat VIP memberships.
The setup
We used a custom desktop PC for each benchmark. It features an Intel Core 2 Duo Duo E6600 processor (4M Cache, 2.40 GHz), 4GB of DDR2 800 RAM, a 400GB SATA hard drive (7200 RPM), an Nvidia GeForce GTX 460 graphics card, and a 24-inch widescreen LED monitor (1920 x 1080).
We split off a new 75GB partition for a fresh install of Windows 10 Pro (64-bit). We used build 10162, as that’s the latest one that Microsoft currently provides an ISO for, and we had issues upgrading to build 10166.
We then ran each of the eight tests on each browser, making sure to take screenshots along the way. Please note that if you try to replicate the tests, your results will of course differ because you’re using a different computer.
The browsers
Google describes Chrome Canary as “the most bleeding-edge official version of Chrome and somewhat of a mix between Chrome dev and the Chromium snapshot builds.” We used version 45.0.2454.3; you can download the latest Canary build from here.
As its name implies, Firefox Nightly builds consist of the latest Firefox code packaged up every night for bleeding-edge testers. We used version 42.0a1 (2015-07-12); you can download the latest Nightly build from here.
Microsoft Edge is Microsoft Internet Explorer’s successor. We used version 20.10162.0.0. Because Edge will be shipping on all Windows 10 devices (PCs, tablets, smartphones, and so on), you can only get it right now by installing a preview of Microsoft’s latest operating system.
[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":1766339,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,","session":"A"}']
The results
Finally, the part you’ve been waiting for. Again, please remember that the exact numbers don’t matter: How they compare between browsers within a given test is what counts.
Here’s the rundown (click on an individual test to see the nitty-gritty details):
- SunSpider: Edge wins!
- Octane: Chrome wins!
- Kraken: Chrome wins!
- JetStream: Chrome wins!
- Oort Online: Chrome wins!
- Peacekeeper: Firefox wins!
- WebXPRT: Chrome wins!
- HTML5Test: Chrome wins!
If you’d prefer to see all the results on one page, please click here.
Final thoughts
While Chrome is the clear winner here, neither Firefox nor Edge performed particularly poorly. The fact of the matter is that browser performance is improving at a very rapid pace, and it shouldn’t be your only consideration when picking what tool you’ll use to consume Internet content.
[aditude-amp id="medium2" targeting='{"env":"staging","page_type":"article","post_id":1766339,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,","session":"A"}']
The good news here is Windows users look like they’re getting a solid new default browser. We’ll give these tests another look soon after Windows 10 arrives at the end of the month.
SunSpider is a JavaScript benchmark, probably one of the oldest ones around. We ran version 1.0.2, the most current one available.
For this benchmark, a lower number is better. From previous tests I’ve performed, I’ve seen much lower Sunspider numbers. I suspect these results are due to a combination of older hardware, Windows 10 still being unfinished, and of course the fact these are beta browsers.
[aditude-amp id="medium3" targeting='{"env":"staging","page_type":"article","post_id":1766339,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,","session":"A"}']
Chrome Canary started off OK:
Firefox Nightly was noticeably slower:
[aditude-amp id="medium4" targeting='{"env":"staging","page_type":"article","post_id":1766339,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,","session":"A"}']
Microsoft Edge beat them both:
This is the only test Edge won, and it did so in style.
[aditude-amp id="medium5" targeting='{"env":"staging","page_type":"article","post_id":1766339,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,dev,","session":"A"}']
Octane is a JavaScript benchmark developed by Google. We ran the latest version, 2.0.
For this benchmark, a higher number is better.
Chrome Canary set the bar really high:
Firefox Nightly came close:
Microsoft Edge couldn’t beat either of them:
Given that Octane was developed by Google, it’s really no surprise Chrome won this test.
Kraken is a JavaScript benchmark developed by Mozilla. We ran the latest version, 1.1.
For this benchmark, a lower number is better.
Chrome Canary was quick:
Firefox Nightly was a bit slower:
Microsoft Edge was as slow as a snail:
Despite the fact that Kraken is Mozilla’s benchmark, Chrome still won this test.
JetStream is a JavaScript benchmark developed by Apple. We ran the latest version, 1.0.1.
For this benchmark, a higher number is better.
Chrome Canary again set the bar high:
Firefox Nightly couldn’t keep up:
Microsoft Edge couldn’t either:
Looks like Chrome knows how to take on Apple’s benchmark.
Oort Online is a WebGL benchmark created by the developers for the game of the same name. We didn’t see a version number.
For this benchmark, a higher number is better.
Chrome Canary killed this visual benchmark:
Firefox Nightly tried its best:
Microsoft Edge had a tough time:
When it comes to graphics, Chrome wins.
Peacekeeper is a general browser test developed by benchmark maker Futuremark. We didn’t see a version number.
For this benchmark, a higher number is better.
Chrome Canary did reasonably well:
Firefox Nightly did better:
Microsoft Edge couldn’t match either of its competitors:
This is the only test Firefox won, and it did so comfortably.
WebXPRT is an HTML5 and JavaScript test developed by benchmark maker Principled Technologies. We ran version 1998.
For this benchmark, a higher number is better.
Chrome Canary again set the bar quite high:
Firefox Nightly couldn’t match it:
Microsoft Edge couldn’t either:
This benchmark took the longest to run, but Chrome still ended up on top.
HTML5Test is exactly what its name says. We saw no version number, though a perfect score is 555.
For this benchmark, a higher number is better.
Chrome Canary loves HTML5:
Firefox Nightly does as well:
Microsoft Edge is getting there:
Nobody got a perfect score, though Chrome came the closest.
Browser standards are a moving target, though this benchmark does show that Google is pushing the envelope.