Monday, July 26, 2010

The Art of Reviewing Sports Games

This week we’ve got what just may be out best episode yet of Jumping the Shark. First off, we spend nearly an hour talking with Jon-Paul Dyson, the director of the International Center for the History of Electronic Games and Vice President for Exhibit Research and Development at the Strong National Museum of Play with which ICHEG is associated. There’s some really good stuff in there.

After that, however, we get some full-on Abner rage; not over the problems with NCAA ‘11’s Dynasty mode (although we do talk about that), but how not one of the reviews listed on Metacritic mention them. I’m not going to elaborate much on that, but it’s good stuff. What I do want to get into is just what a pain in the ass it is to review sports games.

It really is. I hate reviewing sports games. I’ll do it just for the sake of having something to review and because I don’t think I completely suck at it, but sports games may just be the least fun games to have to analyze from a critical perspective.

I mean just reviewing the on the field/court/ice content is challenging enough because if you’re not a flat out expert in the represented game somebody is going to know more than you and tell you about six different problems you either didn’t see or didn’t care about.

Despite the sporting nature of this blog, I actually know relatively little about sports. I like sports. I know the rules, the positions, team composition, and all that jazz. But if you put me in front of a basketball game, I’m not gonna be able to tell you if it plays a *smart* game of basketball. I have a decent sense of what a good hoops game feels like, but I can’t tell you if they have the 76ers playing the proper offensive or defensive sets. I can’t tell you if they have the right backup center coming in for the Bobcats or if he’s setting up where he should in the paint.

And even if you do know that stuff, the presence of gameplay sliders that affect vitually every single aspect of how a sports game plays, makes any quick-look impression moot. Are NCAA ‘11 sack totals too high? Sure. Can you fix it with a quick slider adjustment? Absolutely. Do I know I can fix it with a quick slider adjustment without putting an hour into every single tweak of each slider? Hell no. And that’s the easy stuff to look at because it comes out just from playing the game.

It gets so much more complicated when you have to start thinking about all the poorly-documented off-the-field stuff these games have. When I reviewed MLB: The Show this year I was at complete loss for how to talk about this stuff. I only have so many hours to play a game. I can’t put 5-10 hours each into Road to The Show, Franchise, Online Franchise, etc. It’s just not doable. Does Road to the Show break down after five seasons? I don’t think so, but I haven’t got a clue. And if it does, and I’m playing as a Pitcher, that doesn’t mean if I play as a SS I’m going to have the same problem.

Baseball is stats heavy as it is, but it goes so far beyond that. Am I really going to test how well the AI manages three levels of minor leagues where players can take five years to develop into major leaguers? How do I begin to know how TV contracts affect the league or how improving my facilities reduces my team’s injury rate? The permutations of things that can affect or even ruin the experience is so long as to be ridiculous.

Now, all this is not to defend the plethora of NCAA ‘11 reviews that completely missed that by year two of a dynasty half the best teams in the nation or running out kickers who struggle to make extra points. The fact that teams in Dynasty mode fall apart three to five seasons down the road is not that difficult to find with just a little effort. It’s just that it requires different things of a reviewer to find this stuff than does reviewing other types of games. In most other genres you just play the games. In sports games you have to spend hours upon hours just running tests, looking at stats, comparing them to online stats databases.

In other words, you have to spend time researching instead of just playing and I think that’s where a lot of reviewers, and the editors who assign them, fail when it comes to handling sports games. It’s not fun work to review these games and the time investment to do it is probably not worth what a reviewer is getting paid to do it, if they’re getting paid at all. And even for the guys (or gals) who do a credible job of attempting to thoroughly play and test a sports game, you are sure to miss something big that someone buying and playing the game for fun will catch inside of a week. There’s just too much stuff.

It’s tempting to look at sports games, something that has a real world analog, and say that it should be easy to analyze them and come up with an accurate, thorough piece of criticism. In reality, though, I think sports games are among the most difficult genres to review. The only genre I can think of that I find more challenging is strategy games and that’s mostly because I’m an idiot who struggles to beat Civ IV on the Prince difficulty. But that’s a post for another day.


Jayhawker said...

Great post, Todd.

I think there is one more problem with sports reviews in today's market. I suspect that 90% of the people buying these games don't care, and would hate to wade through a review that is as deep as many of us think should be required.

In short, I don't think there is a market for the kind of reviews we are looking for in the mainstream gaming press. While we rage against the machine in regards to progression and roster management AI, most people just want to know if it will feel good to play.

NCAA 11 passes the smell test with flying colors. 90% of the people buying the game will never notice most of the problems the game has.

Most of the sports gaming public would be happy with RBI Baseball, Madden 95, and NHL 94. Despite the pretty graphics, the average gamer still only expects the game to play as smart and authentic as those classics.

I see most of the reviews that make up Metacritic as actually doing a decent job for that demographic. So I've never been too worked up about the mainstream press reviews, because I never felt like they were written for me. And before blogs and forums, I was content to use Usenet as a way to find like-minded sportsgamers that were willing to dig into games and come up with the kind of assessments I was looking for.

That said, I think it is time for OS to get a little more serious about their reviews. Some are good, but too many are just barely more in-depth than what you can find on IGN or Gamespot. But even OS has trouble differentiating between types of sportsgamers.

While every time I read a complaint about what color cleats a team wears, or if players shake hands after games, I feel the urge gouge my eyes out with a fork, there is no doubt that they make up a significant portion of the OS demographic.

Sportsgamers, more than any other genre, really have to just take what they can get. Hopefully most have figured out how to find what they want.

todd brakke said...

Glad you liked the post!

I don't disagree with anything you wrote there. Many, possibly even most, NCAA buyers won't be concerned with some of these issues. I also don't think a reviewer who finds and mentions them has to dock the game big points over something like the Dynasty mode flaws. That's totally up to the reviewer. But for so many reviews en mass to not find/mention the Dynasty issues at all is pretty bad. :)

jcalvert said...

This is a great article; in fact I enjoyed it so much I just linked to it as a supplement to a recent post.

Reviewing sports games is a thankless task. You could spend 25 hrs playing a game prior to a review, and still miss a couple of minor (to you) issues that a number of gamers consider show stoppers. What is a poor sports game reviewer to do?

The problem with the majority of the reviews currently posted is that the reviewers probably do not have endless time to dedicate to a review, so they play the game, sim a few seasons, and call it a day. This is probably good enough for the majority of casual fans, but not for the folks that read your blog!