Does game quality translate into better financial performance?

John Riccitiello, chief executive of Electronic Arts, came into EA a couple of years ago with the commitment to improve the quality of the big company’s games. During his calls with financial analysts, Riccitiello regularly cited average review scores for the company’s games.

That raises the question of whether there is a direct relationship between the quality of a game to the bottom line. Metacritic, which aggregates review scores by a bunch of established game reviewers, has become the defacto measure of a game’s quality. It isn’t a perfect measure of quality, but it’s the best the industry has. The thinking is that a high-quality game gets good review scores, which translate into a high Metacritic average score. That, theoretically, should lead to better sales and an improved bottom line.

“With games, you see a correlation between a bad review and poor sales,” said Michael Gartenberg, an analyst at Interpret. “That’s different from movies, where some of the best-selling movies get terrible reviews because the critics and the audiences are out of sync.”

As the industry heads toward another E3 trade show next week in Los Angeles, I am almost certain the subject of game quality will come up at every press conference and every interview that I do. There are still too many console games out there, far more than any fan can play. There should be a correlation of game review scores (as calculated by EEDAR, which uses a similar method to Metacritic) and financial performance, but the chart below shows it isn’t universal.

Just about every CEO brings up quality these days as the way to stand out from a pack of also-rans. This week, on a conference call, Strauss Zelnick, chairman of game developer Take-Two Interactive, cited the need for quality improvements as a reason for delaying two major games this fiscal year. Those delays forced Take-Two to revise its outlook downward. But Zelnick said the delay is worth it if it means that the games will be received better by both critics and gamers.

Pouring money into game quality is a matter of resources. Does the team have enough people and time to get its job done? Riccitiello has said the company has focused its development resources now. It’s making fewer games and is investing what it needs to in each one. When big games are delayed, such as Spore or the Godfather 2, quality is often the main reason. But EA has been losing money because its costs are growing at a faster rate than its revenues. The focus on quality costs a lot of money.

Jesse Divnich, an analyst at market research firm EEDAR, provided us with data that shows the average EEDAR average review scores across the top selling games of five of the major industry publishers: Electronic Arts, Activision Blizzard, Nintendo, THQ and Ubisoft. As you can see, it’s interesting to see the trends.

Both EA and THQ show improvement over the last three years in review scores, but both lost significant amounts of money in the past year, based on GAAP financial results. EA currently has the highest average review scores for its top 10 games. Activision Blizzard saw its review scores drop over three years, but it’s reporting a lower operating loss than either EA or THQ. Nintendo has somewhat flat results for the past three years on review scores, but its financial performance was stellar in the past year.

Ubisoft has had steady improvement in its review scores and it has healthy profits. But its low average score of 76 in 2008 highlights one of the big weaknesses of average review scores. About 32 percent of Ubisoft’s revenues come from casual games, or light web games or titles for girls on the Nintendo DS. Metacritic scores are notoriously inaccurate on such titles, since most of the reviewers are hardcore game reviewers, while the audience is mostly women or girls. Other makers of casual games suffer the same problem.

“I feel kind of torn about Metacritic,” said Graham Hopper, executive vice president and general manager of Disney Interactive Studios. “It’s continual frustration for me. For our audiences, the scores are less useful than I would like them to be. That said, for the hardcore games, we use it and track it. A low-performing hardcore title on Metacritic tends not to sell well.”

Then there are the anomalies that no one can explain. Sega won a lot of attention and critical acclaim for Mad World (pictured top), an ultra-violent but comedic game with a unique art style. It was a rare mature-rated title that debuted on the Nintendo Wii, known for its family fare. On Metacritic, Mad World scored a healthy 82 average review rating, with an average 9.2 out of 10 rating by fans. But it sold less than 70,000 units in North America. That’s a complete bomb.

Sean Ratcliffe, vice president of marketing at Sega of America, said the ideas behind the game were innovative, but it’s possible that the game didn’t come together in a way that absolutely wowed the gamers. Sega still believes in the team that made the game and the strategy, but it accepts that it struck out with Mad World.

Michael Pachter, an analyst at Wedbush Morgan Securities, said that review scores matter a lot to consumers if the games are brand new intellectual property, not based on an existing brand such as a movie game. And any outlier scores, such as a 60 or a 90 out of 100, are sure to correlate with the success or failure of a title, he said.

“Sales of Grand Theft Auto aren’t likely to be affected if they got either an 88 or a 98,” he said. “But it would really hurt sales if they got a 70.”

The Mad World example shows that you just can’t rely on review scores alone to predict success. By and large, it’s important to pay attention to review scores, but not too much. It’s very difficult to move the needle on game quality, even with drastic changes in investment in games. It’s also exceedingly difficult to get game profits and quality moving in the same direction at the same time.

By the same token, if average review scores are dropping as losses are climbing, that’s a huge problem.