If you head on over to the Metacritic page for Natural Selection 2, you might notice something strange. Of the seven reviews listed, only one is scored below an 80.
Also, it doesn't actually exist.
Last week, GameSpot reviews editor Kevin VanOrd pulled his site's review of Natural Selection 2. The review, scored 60/100 and written by a freelancer named Eric Neigher, had been eviscerated by readers and commenters who pointed out a number of mistakes—for example, the review said the recently-released indie game was $30, when it actually costs $25. Other errors involved the game's engine and load times.
So VanOrd removed the review, citing "several inaccuracies" and apologizing to his readers. Yesterday, GameSpot ran a re-review by a new writer, Ashton Raze, who scored the game an 80.
But the original review is still on Metacritic. A big yellow 60 sits right on Natural Selection 2's front page, warning readers that "disappointing execution holds it back." Clicking "read full review" will take you to a broken link.
So why hasn't the review been replaced? I asked Metacritic head Marc Doyle, who told me that they have a one-shot policy for all reviews and all gaming outlets. (Kotaku is not listed on Metacritic, as we do not use review scores.)
"Yes, the critics we track know—and I spoke to the GameSpot team about this this week - that we only accept the first review and first score published for a given game," Doyle told me in an e-mail. "I'm explicit about this policy with every new publication we agree to track. It's a critic-protection measure, instituted in 2003 after I found that many publications had been pressured to raise review scores (or de-publish reviews) to satisfy outside influences. Our policy acted as a disincentive for these outside forces to apply that type of inappropriate pressure."
(Note: GameSpot and Metacritic are both owned by the same parent company, CBS Interactive.)
"We only accept the first review and first score published for a given game... It's a critic-protection measure."
It's a policy that seems to be enforced with the best of intentions—just about every reviewer in the gaming industry has heard shady stories about publishers trying to get scores changed—but it doesn't account for situations like this, where a writer's mistakes could seriously impact the fate of a game. I asked GameSpot's VanOrd how he felt about this whole situation, but he wouldn't comment on the specifics. "The review had multiple factual inaccuracies that cast a shadow over the entire piece, so we chose to pull it and reassign, which is a rarity, of course," he told me.
"So what?" you might be asking. "Why does it matter what kind of review scores this game has on Metacritic? Aren't review scores just arbitrary anyway? You guys don't use them!"
For better or worse, Metacritic has become an important tool for people in the gaming industry; some publishers even use it to decide whether to give out bonuses, a practice I've criticized in the past. And studies have shown that a game's Metacritic score can have significant impact on its sales.
For an indie game like Natural Selection 2—a game made over the course of six years by a team of just seven people—one mediocre score could be catastrophic. (Although they seem to be doing okay, and Kotaku's Kate Cox had a ton of praise for the competitive game in our review.)
I asked Hugh Jeremy, community manager and jack-of-all-trades over at Natural Selection 2 developer Unknown Worlds, what the team thought of this whole situation.
"We fully respect Gamespot's journalistic processes and are very thankful that they took the time to review NS2. Many critics will not touch smaller indie games with a ten-foot barge pole," Jeremy told me. He said they've reached out to Metacritic for an explanation, but they haven't yet heard back. [Update: After the publication of this piece, Metacritic's Doyle sent me a note to say he had offered to get on the phone with Jeremy, but that Jeremy never got back to him.]
"In general, it is extremely important that score aggregators reflect accurate and timely information about the reviews they are aggregating," he said. "Their scores are used by players, publishers, investors, other critics and even developers themselves to judge the success or failure of a product. Those scores can make or break dreams."
While Eric Neigher, GameSpot's original reviewer, was certainly entitled to give the game whatever score he felt was appropriate, that score was pulled. GameSpot has re-reviewed the game. Should the most important game score aggregator on the planet not update their listings to reflect that?
This isn't the first time that Neigher's reviews have come under fire. In 2010, the developers behind Monday Night Combat publicly questioned if he had played the game before writing a review for 1up. (I reached out to Neigher earlier today to see if he'd like to give his thoughts for this story, but I haven't yet heard back.)
And a glance through some of Neigher's old work shows some rather bizarre choices.
"And if you're all butt-hurt about not having the same stuff as the kids across the Pacific, I hear you, but please believe me when I say that if you allow that butt-hurtedness to prevent you from buying and playing this game, you fail at life and will never have sex. Yeah, I said it. Because it's true. Don't say I didn't warn you," he wrote in his review of Yakuza 3.
We'll continue to follow and update this story as we hear more.