Game reviewers aren't satisfied with the opportunities they get to review games. In interviews, they tell me that things could be better. And they've explained how some standard practices affect the reviews they write.
What if it could all be perfect? I asked a few top game reviewers to tell me their desires for the perfect review experience. I put their requests in a list. The result is the first draft of:
The Game Reviewer's Bill Of Rights
Item 1: A final, boxed copy of a game will be provided to a reviewer prior to the writing of a review
Item 2: The review copy of a game will be made available to the reviewer at least a week prior to a game's release
Item 3: Developers and publishers will not be present while a game is reviewed
Item 4: Reviewers will be given access to a game's online mode during the review process
Item 5: To be determined -- this is a rough draft
Now does any of that sound implausible? Maybe it should. Read on to see what game reviewers and game creators have to say about these issues. And let us know what amendments should be made to this list.
1) A final, boxed copy of a game will be provided to a reviewer prior to the writing of a review
This one is essentially impossible for any print magazine reviewers. They need to review a game a couple of months before its release. But you'd expect it to be a possibility for online reviewers. Former GameSpot reviewer Alex Navarro, who expressed his desire for items #1 and 2 on this list, said this "almost never happens because game production schedules don't really allow for that kind of thing."
This is how former GameSpot chief reviewer and Giant Bomb founder Jeff Gerstmann expressed his desire for item #1: "I think all a reviewer should really have in their disposal is exactly what the person purchasing the game will have. A copy of the game that is the final version of the game and the instruction manual. "
"I think all a reviewer should really have in their disposal is exactly what the person purchasing the game will have."
Providing a review with anything less than final boxed copy leaves the reviewer guessing. Says developer Denis Dyack: "They have to make predictions into the future of what the game will be like." Typically magazine reviewers -- but even sometimes online reviewers who are working just days in advance of release -- will be provided notes on what will change or improve in later versions than the one the reviewer is working with.
Proving Dyack's point, here's part of a note I was sent from Microsoft's Xbox PR team ast week regarding the review disc I received for next week's "Ninja Gaiden II." The note explained that load times for the game may be slower in my build than they should be due to the way the review copies of the disc were burned: "In the final boxed copy of the game, which you will receive once the game ships to retail on June 3 in the U.S., you will not experience any lag in load times." As Dyack indicates, a reviewer must decide if they're willing to predict that future.
Some companies get around this by actually getting boxed copies of games to reviewers early. Navarro said Nintendo does a good job making boxed copy available to press a few days before a game is on store shelves (I can confirm this) but said few other companies are that ahead of things. "If everyone operated on the same sort of wavelength that Nintendo does, reviewers' jobs would be a hell of a lot easier."
2) The review copy of a game will be made available to the reviewer at least a week prior to a game's release
Navarro said this is a pie in the sky request, even though he's asking for it. "Every release is so up to the wire that there's never really that room for the critics to get heir hands on it maybe a couple of days ahead of time if you're lucky."
It's not just reviewers who'd like reviewers to have more time with a game. Dyack says the rush to review quickly hurts review quality. He lamented the result of IGN, GameSpot and other major review outlets, all of whom express the desire to have reviews run as close to a game's release date as possible, operating under this pressure. "The majors, in order to get the traffic and penetration rate for the magazines, have to do the reviews very quickly. [The resulting Metacritic average is] more like a Gallup poll of how people are feeling that month rather than how they feel in the long-term about the quality of the game."
"[The resulting Metacritic average is] more like a Gallup poll of how people are feeling that month rather than how they feel in the long-term about the quality of the game."
Not everyone thinks the rush ruins the reviews. Gerstmann, who once had to binge on "The Legend of Zelda: The Ocarina of Time" starting on a Friday to post his review on a Monday said: "The best game reviewers are ones that can marathon through that stuff but can understand if they let it sink in or simmer over the course of a week or two weeks, how would that experience have differed and they factor that in?"
Still, is getting a game a week in advance of release of when a review needs to be filed too much to ask?
3) Developers should not be present while a game is reviewed
It's become common for publishers to host pre-release review sessions in controlled locations like a company headquarters or hotel room. This is done in lieu of sending out an early copy of the game. At such events, a major game is made available for review, but the reviewer can't take the game home or back to the office. This is how Microsoft handled "Halo 3" and Rockstar handled "Grand Theft Auto IV," just to provide two recent examples.
Sometimes publishers and/or developers will come to a critic's office and request to sit with a reviewer while they start playing the game for review. I thought it was nice to have a representative of Bungie sit in on my play-through of "Halo 3" at a midtown Manhattan hotel and when the creative director for "Assassin's Creed" dropped by MTV HQ to watch me start his game. But I wasn't writing a review of these games; I was just covering them for my reporting and found the special access informative. Had I been reviewing the games, would it have been a problem?
Gerstmann said having developers present is never a good idea. "There was one game that was released last year where the publisher wanted to have the reviewer sit with the developer for the first three hours of the time playing it," he told me. "Every time I hear something like that I think, wow the publisher must really not be confident about the quality of this game."
"Every time I hear something like that I think, wow the publisher must really not be confident about the quality of this game."
Tal Blevins, vice president of games content at IGN is in agreement. "We also prefer to be apart from the publisher when critiquing a game, so reviewing a title in our office or at home is preferred over a closed review session at a hotel or the publisher's office."
Navarro said he thought game creators' presence during a review session undermine the review or the reviewer. "I think the second you let a developer or publisher or PR person look over you're shoulder while you're playing it [for a review] maybe it's just me being paranoid, but it just feeds into this feeling of they're watching how you're playing the game. They're looking for excuses to debunk what you say about it, if you say anything negative about it."
In an e-mail to me, 1up.com executive editor Ryan Scott expressed his frustration about publisher-attended review sessions and took a swing at the justifications those publishers make for them. "We (like everyone else) have engaged in the occasional offsite review when dire timing-related circumstances warranted it, but we've always taken care to disclose this sort of information in the reviews themselves. All told, though, I think it's a shady practice--publishers always cite 'fear of software piracy' as the primary reason for orchestrating these types of sessions, but I've never once had such a publisher honor a request to arrange a 'normal' review alongside a signed NDA, despite our spotless record with regard to credibility."
4) Reviewers will be given access to a game's online mode during the review process
Navarro tells me that this is a no-brainer but still isn't the norm. "Companies are absolutely abysmal at planning for online testing… I can't tell you how many times -- for the same game released every year with online -- we've had to go to [publishers] year after year and say, hey, are you doing some sort of online session for this? Are you doing something where the reviewers can get together and play this?" He said his fellow reviewers at GameSpot would often wait until the retail release of a game, try its online mode against early consumers so they could assess it, and only then run their reviews.
So when is this all going to improve?
Navarro is pessimistic, blaming just about every complaint on the same problem: time crunch. "What I consider to be a perfect review session probably isn't going to exist any time soon simply because of the way the industry works," he said. "I wish the game industry worked a lot more like the film industry or the music industry where there is a hard date, they have the product in hand a couple of weeks ahead of time, and they're actually looking to get the critical scores out there before a game comes out. But that's just not the way it works. People are too panicky, too pressed for time. That's just the way it is."
See any solutions here?
What do you think should be part of the Game Reviewers' Bill of Rights?
Got a comment you can't bring yourself to share below this post? Drop me an e-mail.