tests using a JavaScript benchmark called SunSpider. While the test includes a variety of challenges ranging from 3D raytracing to text manipulation and cryptographic calculations, it doesn’t necessarily predict how well a browser will perform when rendering randomly selected Web pages, Padjorny argues. (It’s not hard to see, though, why Apple’s engineers would favor the test: it was created by the same Apple team that built WebKit, the open-source layout engine at the core of both Chrome and Safari.)
“There are a lot of small details involved in browser performance, and Google coming with Chrome and designing for speed from the get-go, probably gets more of the small details right—that is the primary conclusion from the speed test,” says Padjorny. “But the secondary conclusion is that if you just focus on JavaScript benchmarking, all you are going to do is improve on that indicator. It’s like taking a test that you know all the answers to.”
Blaze provides a free service called “Mobitest” that allows website owners to test the performance of their sites on actual phones in Blaze’s labs. Once it had built the Mobitest facility, the company realized that it could also be used to compare smartphone browsers, Podjarny says. “We said, ‘What interesting insights can we harvest,’ and one of the ones we were most curious about was who is actually faster—is iPhone faster than Android? Now that we have over 45,000 measurements, we definitively know the answer.”
The lesson for Apple in the study results, Podjarny says, is “you should measure your assumptions. We have this tool now. Go ahead and use whatever internal tools you have too, but then test it [on Mobitest] and see if it fares as expected.” And there’s a larger lesson for Web developers: they shouldn’t assume that their sites will perform identically on Android phones and iPhones. “Site owners need to take care to measure performance and understand what their site looks like when viewed through the lens of an iOS device and an Android device,” Podjarny says.
