Everyone hates Metacritic.
Developers despise it. Bloggers think it’s damaging. And gamers? Well, when they aren’t using it to fuel fanboy rage, they want Metacritic to burn to the ground. That’s pretty much everyone, right? I can’t think of anybody that exists outside of those three groups … well, nobody except for the people that actually use Metacritic.
For gamers, developers, and bloggers, Metacritic is just noise. We’re entrenched in gaming’s cultures and subcultures. We know, before a game comes out, if we want to buy it. The reviews only exist to validate our pre-formed opinions. When someone from within our echo-chamber speaks out on Metacritic, invariably they work backwards from another pre-formed opinion: “Metacritic is bad and it’s useless.”
You would think that any good criticism of Metacritic would start by attacking its ability (or inability) to help consumers. If you can prove that it doesn’t help people make good purchasing decisions, then everything else about the site falls apart.
For the unaware, Metacritic is a website that collects and collates review scores from dozens of outlets for music, movies, television, and video games. The site’s editorial team takes those scores and runs them through a proprietary algorithm that generates a final number out of 100. That number is the Metascore.
It’s an incredibly reductive process that torches down criticism until nothing remains but a magical number. Critics hate it for that, but the question is: does the Metascore work?
Does Metacritic’s review gathering and number crunching produce a tool that is useful to consumers? The website’s founder Marc Doyle told GamesBeat that he and his team designed Metacritic for entertainment consumers and that he has known from the beginning that it works.
“Starting out, our own team provided the important evidence [that Metacritic works],” Doyle told GamesBeat. “As we were developing the website back in 1999, we pulled together all the various critic reviews on Excel spreadsheets, calculated what would later become Metascores, and used them in deciding what we should see — and what to avoid. That’s where the idea for Metacritic really solidified.”
Even though Doyle believes in his website he also recognizes that it should only be one tool that consumers consult before making a purchase.
“Even for me, I’ll absolutely see movies, buy albums, and play games with low Metascores because they feature an actor I like or a genre I enjoy,” he said. “But for titles I’m less familiar with, I like to know what I’m getting into with entertainment considering how expensive it has become and how much time a game or TV series can consume.”
It isn’t surprising that Metacritic’s founder believes that it works, but I wanted to look into it myself.
I decided to try to figure out if Metacritic is useful for someone who only has a passing interest in gaming. My hunch is that busy people who don’t spend all their time reading message boards are probably more satisfied with their game-purchasing decisions if they use Metacritic.
That’s a really hard thing to prove. No research exists on the subject, so I’m going to put myself in the shoes of someone using Metacritic as it is intended. The GamesBeat team helped come up with questions that might come up for the average gamer, and I will attempt to figure out answers primarily using Metacritic.
Here is what I found:
I want to spend some money on a PlayStation 3 game. What is good right now?
The first thing Metacritic confronts consumers with is a chart for “new and notable” releases in movies, games, TV, and music. If a person got no further than this, they would see that a game called The Last of Us is listed high above the next closest game.
Above: The top PlayStation 3 games on Metacritic for the last 90 days.
If they dig a little deeper to the High Scores section, which lists the best-scoring games for the past 90 days, they will see a slightly more confusing list that consists of smaller mobile games, downloadable content, and retail games. By selecting just one platform, in this case the PlayStation 3, the list is much easier to parse.
The Last of Us still dominates the list, but I can see other games like Hotline Miami and Dragon’s Crown near the top.
The entire top 10 for PS3, not including DLC, looks like this:
- The Last of Us
- Hotline Miami
- Dungeons & Dragons: Chronicles of Mystara
- Dragon’s Crown
- GRID 2
- Magic: Duels of the Planeswalkers 2014
- Pool Nation
- Metro: Last Light
- Stealth Inc: A Clone in the Dark
- Tales of Xillia
I would have no problems handing that list to anyone looking for a new PlayStation 3 game. Anyone who knows that they don’t like dungeons or dragons and wants a full retail game can quickly squeeze it down to just The Last of Us
Someone who is open to smaller games might even take a chance on something like Hotline Miami simply because it scored so well.
Consumers can quickly expand this list by searching for the best game released so far in 2013. That reveals games like BioShock Infinite, Tomb Raider, and MLB 13: The Show.
Metacritic isn’t perfect, though. These lists include downloadable content, and the site doesn’t distinguish between the add-on packs and the full games.
All I play is Call of Duty and Madden, but I want to try something new and daring. What is good like that?
I think many people believe that Metacritic’s system rewards conventional games over titles that try something new. The site’s tools do not reflect that. Instead, when I navigate to the High Scores for all systems in 2012, Metacritic gives me a list of quality titles that span most genres.
I don’t know what game you would expect to find on top of that list, but I doubt it is The World Ends with You: Solo Remix for iPad. That game has a 95, which makes it the best-reviewed game on Metacritic in 2012. The rest of the top 15 is just as unconventional:
- The World Ends with you: Solo Remix for iPad
- The Walking Dead: A Telltale Game Series
- Bastion
- Persona 4 Golden
- Mass Effect 3
- Punch Quest
- Bejeweled HD
- Cave Story
- Beat Sneak Bandit
- Tiny Wings HD
- Journey
- Xenoblade Chronicles
- Mikey Shorts
- Reckless Racing 2
- Dishonored
Almost half of those titles are iOS games, but I think the most interesting item on the list is The Walking Dead. Telltale’s horror adventure title is the highest-rated console and PC game for all of 2012.
If Metacritic truly rewarded safe games that didn’t take any chances, then The Walking Dead wouldn’t be anywhere near the top of the list. The truth is that reviewers almost universally rewarded the title for attempting something new and for having the skills to execute the design.
This is exactly the game I would want under-informed consumers to learn about and try. If they use Metacritic, then it is very likely that will happen.
If The Walking Dead doesn’t sound good to them, then they could end up trying Bastion, Person 4 Golden, Cave Story, Journey, or Xenoblade Chronicles. These are all beloved games that in no way reflect our industry’s obsession with military shooters. In the battle to expand the audiences for different kinds of games, Metacritic fights for the good guys.