11.30.07
3.14159265897932384626433832… How many numbers is too many?
Posted in Uncategorized tagged corruption, film, journalism, ratings, skew, stars at 1:13 pm by supereffective
As you’ve noticed, I use a 4-star scale to review games. This is a conscious decision… I could use a 5-point scale, or a 10-point scale, or letter grades, or Terror-color-levels, but I choose to use 4 stars, with half-star increments. I started this way because that’s how the paper I wrote for in college operated, and I quickly became accustomed to and comfortable with the prospect of having exactly 9 scores I can possibly give. Of those, I believe I’ve used 7. I’ve never given 4-stars, and I’ve never given .5-star. (Before you ask, yes, I’ve given a game a zero. It was MTV’s Celebrity Deathmatch for the PS2.)
The 4-star scale is a beautiful thing. Why? Because video game sites don’t use it. It is the domain of film. It has not been subjected to the years of point-nudging that has caused the acceptable range of scores to fall between 6.0 and 10. (Penny Arcade’s newspost for today shares my concern for this statistical skew.)
Look through your city’s independent news-and-entertainment-weekly and you’ll see the film reviews as a mass of 2/4s. This is proper. Average films are aaaaaaaaverage. Average films do not get an automatic 7/10 just because that’s how the ratings are skewed. That being said, an “average” film that receives a 2/4 is still probably a worthwhile film to see, especially if you like the actors/topic material/director/shininess. Would you play a game that got a 5/10 on GameSpot/IGN/what-have-you?
I will give games 2-star ratings. This doesn’t mean that I find it abysmal, nor does it mean that something’s broken and irreparable. It just means that it’s an average game. It’s your I-Ninja, your La Pucelle Tactics, your Megaman ZX Advent (though I haven’t written the whole review up yet). These are respectable games, but they’re not -superlative- games. They don’t deserve a 7/10, because they’re not 2 whatevers better than average. They are the average. There are better games, and there are worse games; there are far better games, and there are far worse games.
Under the current regime, who’s to say the difference between a 2/10 and a 3/10? When you only have 9 scores you can possibly give out, each one has to mean something. Each one represents a distinct stratus of quality. What’s the difference between 1.5-stars and 1-star? I’d say it’s about the difference between Ontamarama and P.N.03. The former was a moderately fun and kinda unique music/rhythm game that suffered from a lackluster tracklist and absolute dearth of multiplayer; the latter was a hip, stylized 3rd-person shooter with a passable plot and decent graphics but horribly clunky controls and counterintuitive camera, which really hurt when you’re relying on stealth.
Further, since there are only 9 possible scores, the reader will have to investigate the text of the review (where actual points can be made) rather than going off of a number in a box. Why did this game get a 2.5-star review? What was good and bad about it that it should be adjudicated a half-star better than average? Is it the controls? The plot? How were the graphics?
Anymore, the discourse among gaming reviews has been that even a game that gets a 6/10 is “bad.” How bad? Watch the Jeff Gerstmann video-review of Kane and Lynch over on Youtube (as Gamespot have kinda dumped it from their site). A 6/10. Skewed like lamb shishkabob, yo.
Heartslaught said,
November 30, 2007 at 4:59 pm
I can agree whole-heartedly here. The fact that the video game industry skews ratings so bad is the reason I have stopped reading them for the most part. I have mostly relied on my own experience or what friends have told me about games. Why?
Because I know that money and politics aren’t involved. It’s the reason I read this blog. It’s just someone who has a fantastic gift of description describing for other people what the games he plays are like in the most biased fashion he can possibly muster. Why? He loves his games and what makes each individual one unique.
What does this have to do with scoring systems? Everything. I’d rather get my information from an unbiased source than someone who writes a review based on his paycheck. Or a group that cans one of their best (who was making an honest to god review… with most likely a skewed score becuase of the reasons listed above in the blog post) due to the fact that the game he gave a bad score to said game that was being heavily advertised on the site he worked for.
One, I hope he finds better work and pay elsewhere, because it’s one hell of a reviewer. Two, there may not be a shoehorn big enough to get said stick out of said company’s ass.
If anything… I hope this makes the gaming community think about what is going on in the review sector of the community, and ways to possibly change it for the better.
chesh said,
December 5, 2007 at 11:50 am
This is not related to the above (very important) issue, but it seems like something you’d appreciate: http://flickr.com/photos/twystneko/1917199532/in/set-72157601866584021/
Kee said,
December 5, 2007 at 5:31 pm
After Galaxy, you’re not tired of stars?
Kee said,
March 24, 2008 at 10:10 am