PunditWatch: Interesting, fun, but slightly flawed

We’ve been noticing a lot of trackbacks (links coming back to our stories from other sites) recently from a site called PunditWatch. As most of the stories with these trackbacks are mine, I’ve been checking it out fairly regularly.

Here’s what it does. Whenever we write something about a certain topic (Apple products in particular), PunditWatch looks at what we wrote and extracts a prediction for a new product, technology or deal. When the time period placed on that prediction is up, PunditWatch determines if the prediction was right or wrong. This effects the sites’ or people it follows’ (called “Pundits”) stock price (its way of keeping score).

The service, which officially launched today under its parent Hubdub.com, is fun. Fun, but flawed. (And I say that knowing full-well that we are currently number three on its Pundit Leaderboard.)

You see, a lot of the predictions PunditWatch has us making are simply us reporting rumors but not saying definitively one way or another if we believe them. I supposed you could take us writing about something as a sign we think that it’s true, except in the circumstances where we report both sides. That happens to be the case with our current “Settled Predictions.” We have a 50 percent correct rate because according to PunditWatch we predicted both “Yes” and “No” as to whether Google and Yahoo would announce an extension to their search advertising deal before the end of May.

Again, that is because we wrote about the story from both sides based on information at the time. If I were to make an actual prediction on it, it would have been complicated. I would probably say something like: “I think Google and Yahoo will extend their search advertising deal provided that Microsoft doesn’t buy Yahoo’s search business and/or Carl Icahn doesn’t takeover Yahoo and/or the government doesn’t start investigating such a deal. Sure, I could go out on a limb and make a definitive prediction, but with something like that there are too many variables.

On the other hand, when I do got out on a limb and make a prediction, such as when I said that Apple would not launch the 3G iPhone in May despite many reports suggesting it was likely, PunditWatch didn’t exactly pick that one up. Well it kind of did, but in a more general way. It has us down for a predicted announcement date of June 1st to the 14th, which is good since it will probably happen at WWDC on the 9th — but that’s a pretty easy one, I would have rather gotten the points for saying it would not come out in May when others were saying it would.

The service also has us down for an announcement between June 15th and the 30th, which I didn’t say — but I do believe that is when it will be actually released (after being announced at WWDC).

I see some problems in other future predictions the service has us down for. It has us prediction that Microsoft will buy Facebook before the end of 2009. Certainly I believe that Microsoft wants to, and I’ve written about that, but I happen to believe that Facebook will not sell to them, at least not this year — which I’ve also said.

So while I like the idea behind PunditWatch and enjoy some of the predictions it extracts from my articles (such as that the majority of top 25 Twitterers will not be techies by May 2009 — which I’m up in the air about right now), it’s a little too hit-or-miss on what I’m actually predicting versus what I’m just putting out there as a rumor.

If PunditWatch would let me manually tweak these predictions to what I actually believe, I’d be happy to do so. I’m very confident in my prediction abilities. Check out my predictions for 2008 on my personal blog; I’d say I’m doing pretty well so far.