Traffic measuring continued: Why Compete doesn’t work, and why Quantcast does

It sucks when your Web site’s traffic isn’t being measured correctly.

It also sucks when you’re trying to measure the significance of someone else’s site, and are getting conflicting signals.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":3429,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,","session":"B"}']

Here’s what we’ve learned over the past few days, after our initial piece on the problems of Alexa, Quantcast and Compete, all sites that independently verify how much traffic a site is getting.

We’ve learned that if a measuring company doesn’t have a tracking pixel directly on the Web site it is tracking, the numbers are extremely unreliable.

This may be nothing new for insiders, but the public data provided by Alexa, Quantcast and Compete can be misleading for the rest of us, because they purport to be accurate. In fact, that data is almost all collected from the surfing patterns of a very small sample of people. Those people are not going to be surfing certain niche sites as much as the general population.

Take the example of Streetfire, a very popular video site for car enthusiasts. Until our initial post, co-founder Adam Bruce was somewhat content with Google analytics, which showed his site had three million unique visitors a month (he let us login to his Google account to confirm). This matched information of his server logs, which Google also manages, and it was compatible with the ad server tracker from Adjuggler. Of course, this was all internal information.

Outsiders had access to Alexa (see its Streetfire rankings), a free site measuring tool, but Bruce believes most people recognize Alexa’s shortcomings, so he isn’t taking Alexa too seriously. Then along came two more sites, Quantcast and Compete, saying they can do a better job than Alexa. After we pointed to them in our post, Bruce checked them, but he found they were both showing his traffic at about an eighth to a twelfth of his internal numbes. Alarmed, and filled with renewed doubt about the reliability of his internal numbers (because server logs can be unreliable too, as can Google Analytics), he called up Compete and Quantcast to find out more.

Notably, both companies cooperated. Quantcast responded by offering him a tracking pixel, to be put directly on his site. This opened Quantcast’s eyes: It saw Streetfire’s traffic was indeed much higher than it had estimated, and told Bruce it will readjust its numbers after a full month’s reading. Compete, however said it couldn’t deliver a tracking pixel — in part, it said, because Nielsen/Netratings owned patents on this sort of thing, and that Quantcast can get away with it because it is too small to sue. (Although, we’re not certain whether Netratings really does have a patent on this).

To be fair, Compete does have impressive data: It says it gets data from the surfing habits of two million people, which is more than comScore, Hitwise, Nielsen and Alexa. It takes extensive measures to ensure data is diverse — gathering it from multiple Internet and application service providers, panels and its own toolbar — all this is more than can be said for the others. Compete has an impressive list of testimonials, but read through the list — most of those people aren’t experts, and even if they are, our hunch is that they wrote positively about Compete, just like we did initially, because it looked so much better than what Alexa was offering.

All these strengths of Compete, however, serve to underscore our point. If it really has such a industry-leading sampling, why did it go so horribly wrong with Streetfire? We called up Compete, and asked. We talked with product manager Jay Meattle, and Stephen DiMarco, who is in charge of “emerging markets.” They said they went back to their panel sampling and studied why a video site like Streetfire would be undercounted. Their analysis found video-oriented sites like Streetfire were being undercounted by a factor of four. Adjusting their numbers, that put Streetfire’s monthly uniques back up to 1 million or so, but not at the 3 million the other trackers were showing. In other words, it is still undercounting — and it came only after Streetfire requested a revision.

[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":3429,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,","session":"B"}']

This is particularly worrying because Streetfire is a major site. If a car enthusiast site of 3 million uniques is being undercounted by the biggest objective panel out there, then well, the rest of us little chickens are hosed. The Compete guys told us they are more interested in serving general consumer sites (their main customers), and don’t really care about niche sites like say, VentureBeat ;). “Our clients are big consumer marketers,” they said. Fair enough. They take Streetfire seriously, they said, because it is a big site. But that also leaves us with this hunch: Most consumer sites are overcounted by Compete. Both may have an interest in that being the case.

Meanwhile, Adam Bruce, of Streetfire, is relieved because Quantcast’s tracker, at least, vindicated him. He can still trust his internal numbers. Bruce’s conclusion: Quantcast offers the best of both worlds. It offers you all the third-party data it can get (see screenshot at bottom; it shows gender, geographic and other data that Bruce says coincides with his sense of his readers), but if you ask it to, it will put a tracker on your site, to verify. Now, not all sites will allow Quantcast do to this. But its livelihood is on the line, like Streetfire’s small team’s is, it makes sense.

Conclusion: When you go to Quantcast for data on a site, be sure it has a tracking pixel on the site before drawing conclusions.

Finally, a note on Site Meter. Bruce said that company’s stats make sense since they are in line with his other sources, though Site Meter doesn’t report unique visitors, just visits.

[aditude-amp id="medium2" targeting='{"env":"staging","page_type":"article","post_id":3429,"post_type":"story","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"business,","session":"B"}']