Want real foresight? Stop polling, start predicting

Now that the 2012 presidential race is on the books, let’s not indulge hindsight — “Oh, we all saw that coming” — and instead take a look at where we were just 48 hours ago.

Polls depicted margins that were razor-thin. “Dead heat!” we heard from CNN over and over. “Too close to call,” trumpeted USA Today.

And august polling organizations supported this view. The Gallup Poll, for example, showed an evenly divided electorate ever since summer. Yes, there was that bump for Obama after the “47 percent” comment and a day in the sun for Romney after that one-sided first debate. But after that, it was Frazier vs. Ali.

For news organizations, the dead-heat perspective was interesting and profitable. As long as the polls were seemingly neck and neck, the 24-hour news channels and news websites had a story to report, so they quoted the polls all the more.

Of course, the electorate told a different story. The outcome wasn’t razor thin, not in terms of its ultimate outcome as seen in the electoral college or even in terms of the popular vote.

On Monday many pundits were conceding this fact. Voices on NPR and other outlets conceded that Obama had many paths to victory and Mitt Romney simply had too few. (Romney himself seemed to know it, given the wistful nature of his final campaign appearances.)

Normally we’d forget all the bad prognosticating, chalk it up to the heat of the moment, and pack it in for another four years.

But perhaps this time we should not. Of all of the outdated electoral institutions — more than the electoral college, more than rusty voting machines — polling has got to go.

Instead we should use modern methods that tap the predictive capability of social networks. I don’t mean Facebook, but the networks we carry around with us. Let me explain.

For more than 20 years, we’ve had a method for measuring public sentiment that has dramatically outperformed polls: online prediction markets, where people can place bets, often with real money, on the outcome of the future events, ranging from the Oscars to elections. My own startup uses this method to forecast television ratings, box office revenue, and advertising effectiveness.

Markets outperform polls, simply put. And while one may see that as a tendentious claim for my own company, we were actually only one of relatively few organizations that showed a decisive Obama lead.

The whole way. Ever since the beginning. Without exceptions.

The Iowa Electronic Markets, which started in 1988, first projected an Obama victory as early as a year ago. The probability was at 80 percent for Obama after the “47 percent” comments, and Romney’s probability was never higher than 40 percent, even after his strong performance in the first debate.

We saw a similar pattern at Intrade, an illegal offshore prediction market where people can bet their own money on future events. Obama took a lead in January, and again Romney’s probability never approached the 50 percent mark.

At Media Predict, we started paying participants to engage in real-world money trading starting in May. Within the first hour of trading, the likelihood of an Obama victory shot to 70 percent. By a 2-to-1 margin, participants bet that Obama would win. Many were Romney supporters, gritting their teeth even as they bet on the other guy, who just seemed to be the better investment.

Of course, it’s easy to see why markets made the right call. Ask people their individual feelings in a poll or survey question, and you get handwringing over taxing the rich or rage over Obamacare. Emotion creeps in.

But ask people to bet real cash on what they think will actually happen, and suddenly you get straight answers.

It’s not just markets; other predictive methods are more effective as well.

As we saw in a recent New York Times article featuring the work of Justin Wolfers, a professor at the University of Pennsylvania, asking people simply, “Who will win?” almost always produced a better election forecast than asking people for whom they will personally vote.

Nate Silver also turned in another great forecast by interpreting polls predictively and not, á la cable news channels, taking their readings as electoral gospel.

So, in light of yesterday’s result, one might suggest a new platform for gathering intelligence about the outside world, whether you’re a political party or Coca Cola vetting your next ad campaign.

Prediction taps social networks. No, I do not mean Facebook by evoking that term, at least not specifically. But prediction does tap predictive social intelligence, not asking people what people feel, but asking them to predict what people in their social network will do.

After all, if you ask me how I feel about something, you will often get unreliable Hamletizing. Neuroscientists are increasingly demonstrating the ways in which the brain — far from being a book that is easily opened, easily read — is much more like a seething cauldron of competing impulses. A part of me might want to go to the gym, and another part of me might want to order takeout and watch Here Comes Honey Boo Boo. Which side will win?

Such mental skirmishes are a regular part of human life, and approval-based polling parades into this warfare and hopes to emerge with a straight answer. Usually it only gets the impulse of the moment.

Ask me to predict something, though, and you draw not from my individual feelings, but from my knowledge of what will happen with my friends, my family, even people I pass on the street. In the laboratory of my mind I can simulate what the most likely outcome will be, considering all of these social factors and without placing my own feelings at the center of my thinking.

If you ask such a predictive question in the context of a marketplace, then market prices will aggregate my best views along with everyone else’s. Best of all, if I make a correct bet, I’ll win money, and if I make a bad bet I’ll lose it: the ultimate truth serum.

The markets told the truth last night. Indeed they’ve been getting it right since 1988. In light of their latest triumph, the time has come to stop the polling and start the predicting.

Brent Stinski is CEO of Media Predict, which uses prediction markets to forecast television ratings, box office revenue, and advertising effectiveness.

Top image courtesy of Everett Collection