I have over 70 hours clocked into Fallout: New Vegas. I can see the final storyline developing but have no idea how it will actually turn out, and judging from my long list of open quests I'm nowhere near triggering the finale. As I continue experiencing just how insanely huge New Vegas is, I wonder how any of the professional reviewers managed to play enough of the game to issue a definitive statement as to its quality.
I tried to read all 77 critic reviews of Fallout: New Vegas on Metacritic in order to see how many of the writers actually finished the game, but I could only get through 26 before I almost died from boredom. Of those 26, only one reviewer stated that he finished the game. Another five reviewers suggested that the game was finished and mentioned how many hours the reviewer logged before writing the review, but the other 20 reviewers gave no indication as to how much of the game they played — and some of these reviews were from fan sites that were certainly under no pressure from anyone to review New Vegas quickly.
He who publishes the first review wins. You get more page hits, or sell more magazines, or just get more acclaim, because in the end most of our readers just want to see the review, and they want it yesterday. Sometimes the publishers ask to see reviews stupid early, as in well before the game is even released, so that they can print review pull-quotes on the game box. That's money in the bank for a video-game journalism outlet, in the form of good will with the publisher and increased chances for more early access down the road.
For the most part this system seems to work out OK in terms of producing credible reviews, as the majority of game campaigns clock in between 8-10 hours. That's pretty easy for a professional reviewer to run through. The system gets mildly problematic when it comes to multiplayer titles, as review events or pre-release, press-only multiplayer rooms just can't replicate the real world conditions by which any multiplayer game should be judged. Role-playing games, however, seem entirely inappropriate to review the way we review everything else as the best RPGs are deep, sprawling adventures with huge storylines and tremendous casts of supporting characters.
It's not as though we don't agree that different genres require different methodologies when it comes to how we write about them. Massively is a viable web property precisely because MMOs require a lot more time to delve into, and constant revisiting, in order to cover effectively. RPGs lack the complex multiplayer layers that an MMO offers, but otherwise they’re very similar genres mechanically.
The wraparound story for an RPG is very similar to the endgame scenarios in an MMO, and would you trust an MMO review that didn't discuss the endgame? Likewise, can you imagine if, at the end of a 60-plus hour RPG campaign, the whole story fell completely apart? Just how strong would the gameplay have to be in order to excuse dropping the ball at the end like that?
Mass Effect 2’s story ended in a disappointingly lukewarm fashion, and if I were to pen a review of the title, that would not escape my analysis. How does Fallout: New Vegas end? Do all of the faction-specific reputation mechanics actually matter, or will we get a similar climax no matter who we’ve favored and who we’ve pissed off? Isn’t that something you’d like to know before deciding that you want to delve into the Mojave Wasteland and commiting the huge time investment the game requires? I'm not suggesting that the quality of the narrative is a make-or-break proposition…but isn't that kind of important for a genre that depends so heavily on its stories?
I asked some colleagues how long they had played New Vegas before writing their reviews. The average time spent was about 25 hours. On the one hand, as someone observed, if reviewers of other media had to devote that many hours every time they reviewed something, they would either demand raises or quit. Book reviewers apparently can have the text for weeks before the review is published. On the other hand, anyone who writes a game review is claiming expertise and authority by doing so. Anyone paid for the job should probably do it right, i.e. they should finish every game they review.
When it comes to RPGs this probably isn't an option most of the time…but I think this is a case where the publishers might be amenable to changing the rules of the review game. The more time the reviewers have with a title the fairer a shake they can give it. That clearly serves the publishers' interests. It just happens to also serve the interests of our readers, as well.
Dennis Scimeca is a freelance writer from Boston, MA. He has written for The Escapist, Gamasutra, and @GAMER magazine, and maintains a blog at punchingsnakes.com. Follow him on Twitter: @DennisScimeca. First Person is his weekly column on Bitmob concerned with meta questions around the video-game industry and the journalism that covers it.