Startup Shootout: Which e-commerce startups perform best on mobile?

Editor’s note: Keynote Systems’ Startup Shootout Index provides some insight into the three-screen challenge now facing anyone with a web presence. It’s the first website performance index to measure load times and completion percentages on desktops, smartphones, and tablets simultaneously. VentureBeat is Keynote’s exclusive media partner, so we’ll be bringing you a fresh set of data from Keynote every month. Check out previous Startup Shootout results.

The holidays are upon us. Now is a perfect time to look at how the online retailers on the Keynote Startup Shootout Index are performing, and see if they are ready for the holiday rush.

We track a number of the leading, startup retailers – Fab.com, Gilt Group, Ideeli, Living Social, One Kings Lane, RueLaLa and Trunk Club. We particularly wanted to look at their performance on smartphones.

RueLaLa was the clear winner, with its performance ahead of the competition across every device.

One Kings Lane does not fare so well, though there may be a good reason for the slower times to a complete page load.

For fast load times, we always recommend limiting what loads “above the fold” so that the user sees content quickly, even if other content “below the fold” continues to load.

But this does raise an important design challenge. Some website owners have customers who would rather scroll down one page rather than click on links. In the case of One Kings Lane, this looks to be the compromise they make.

While the complete page load (often described as “end-to-end” page load time) takes a long time, visitors to the One Kings Lane mobile home page do see the content above the fold very quickly.

But RueLaLa definitely has the fastest “end-to-end” page load times on smartphone browsers. Even on the impressive RueLaLa site, we see 38 new requests being made as the site loads.

This is more than we would recommend.

Keynote suggests no more than 20 requests for optimum speed, which is considered to be a three second or under load time. In addition to the large number of requests, it takes RueLaLa about two seconds to load the base content for its home page, with the corporate image being one of the first items to load.

So, not only is RueLaLa loading too much content, but the core HTML file for the page is taking longer than it should. Even though RueLaLa is the leader for retail sites in the Startup Shootout Index, these are areas it could work on to improve.

Meanwhile, Ideeli takes a tortuous 26 seconds to load its complete home page on the iPhone. It’s loading a large amount of content, and the content that loads early is not useful for initial render of the page. In fact, typically nothing will appear on the browser screen by the three second mark.

Another concern with the Ideeli home page is the large number of domains being used, causing big delays with domain name resolutions. We actually counted 36 separate domains used on the page. For example, we see www.ideeli.com, 0.icdn.ideeli.com, 1.icdn.ideeli.com, 3.icdn.ideeli.com, 0-dam.ideeli.com, 2-dam.ideeli.com, etc.

Large numbers of domain names are less of a concern on the desktop, but on a smartphone, across a less robust network, it quickly causes problems. If each new domain introduces a delay for the DNS resolution request (typically between one-tenth and one-quarter of a second) then multiply that by 36 and you start to see the problem. Extra DNS lookup easily adds up to critical seconds of extra wait time for customers trying to visit the site.

On the Ideeli.com home page, the browser may do a DNS lookup for as many as one out of every four assets on the page.  Many of these are for third-party tags like Facebook tags or Google Analytics, but many are for Ideeli’s own domains or for mobify.com domains (a mobile technology platform that the Ideeli mobile site is built upon). Having to look up multiple domains plus multiple third party tags is really impacting performance.

In contrast, over at RueLaLa we see a number of third-party tags but just one domain from the core site itself. RueLaLa also loads a considerably smaller number of assets – almost a quarter as many as Ideeli.

Domain Sharding – Your Time Has Passed

And one final thought. The divergent experiences we see between RueLaLa and Ideeli bring up another interesting technique that affects performance.

Not so long ago, a web site performance trick called “domain sharding” was popular.

Browsers like Internet Explorer 6 and 7 had limitations that restricted their ability to fetch only two requests from any given domain in parallel at one time. In order to get more requests running in parallel, developers would intentionally use multiple domains, such as image1.example.org, image 2.example.org, and image3.example.org instead of using a single domain called image.example.org.

Hence the term, domain sharding.

Today that limit has been removed with popular browsers now able to fetch six and sometimes even 12 parallel connections at a time to any web server. Domain sharding might still benefit some pages in some conditions, but for mobile sites, the time spent making DNS lookups and making new server connections is often slower than the benefit derived from extra parallelism in the downloads.

If your mobile site is still using domain sharding, it’s time to revisit that decision and consider a different approach.

Keynote tests the sites in the index hourly and around the clock from four locations over the three largest U.S. wireless networks, simulating visitors using three different devices. Data is collected from multiple locations and then aggregated to provide an overall monthly average in terms of both performance and availability.