Sunday, 16 June 2013

the value of ratings

Many people who review games now prefer to rate games using a four points scale - enthusiastic/love, suggest/like, neutral/indifferent, avoid/not for me, as opposed to giving a score out of ten, or a number of stars out of five. I guess this four point scale is something that is clean and direct. You don't need to argue about the definition of what a 7 is and how it is different from an 8. However I don't think it is much more useful than a 10-point scale.

When I see the BGG rating of a game, or it's rank, what it tells me is how much the average BGG user likes this game and how well it is liked compare to other games. Now this concept of an average BGG user is abstract. There is no such real person. If my gaming tastes are close to this BGG average, then I will probably like a highly ranked game on BGG. So the BGG rating and ranking can be tools to guide me on what games I should explore. The drawback is I usually won't bother to check out games ranked beyond 300, or games with a rating below 7. This is despite the fact that there are games that I like which are outside of these thresholds. In this sense, the BGG ratings and rankings does me harm.

Average ratings and rankings consolidate inputs from many people. When a single game reviewer shares his (or her) rating for a game in his review, this is data from just one person. How useful would this information be? It depends on how closely your tastes match this reviewer's tastes. Do you know how well they match? Do you unconsciously take the rating as a sacred number? Oh this game must suck because this reviewer says so. I have a bad habit of scrolling down to the end of a review to see the rating before deciding whether to read the review itself. Maybe that is a quick way to screen out poor and average games, but I think I have missed some good games because of it too. And there is just no accounting for tastes in the first place. Even if I agree with a reviewer on ten games, there is bound to be an eleventh that our opinions differ.

Some reviewers prefer to list bullets points of pros and cons. I find that a little boring sometimes. After a while all the pros and cons sound like they come out of a checklist of possible tags to attach to a game. Is a game simply a list of strengths and weaknesses? Does everyone have the same standards for what are strengths and what are weaknesses? I think in boardgames one man's meat can be another man's poison. And vice versa, of course. Try to imagine a game that you love. If someone then lists all the weaknesses of this game, you may find that none of them matter to you. But if applied to another game that you dislike, you may find that, yeah, all such weaknesses are true and are stopping you from enjoying this other game. In my case, an example would be the unforgiving nature in Antiquity, and that same unforgiving nature in Greed Incorporated. The same applies to strengths of a game. If there is a game you simply don't like, even if someone can list 10 strengths, they wouldn't move you.

Greed Incorporated. I still have not given it a second chance. Not that I'm that strongly against revisiting it, just that it is not a priority.


OK, if ratings and rankings have limited usefulness, and can even be harmful, and pros & cons analyses quickly lose meaning, then what is a reviewer to do? I don't have a right answer. I only have an opinion. I prefer reviews that try to capture the feeling and the essence of playing a game. No need to try to describe the mechanisms and rules in detail. No need to have a comprehensive summary of all key features, as if any omission is a failure. No need to come up with a weakness if you don't feel there is any that bothers you. The same applies for strengths - there is no need to try to be fair and to acknowledge some redeeming quality in a game you dislike. I like it when a reviewer can capture what is unique about a game. Sometimes it can be some central mechanism that drives a game, which is something very concrete. But sometimes it can be just a general feeling when playing the game, which is more abstract and does not require the game mechanism to be explained in detail. I like it when a reviewer explains why a game captures his imagination, or why a game disgusts him. It is something one can relate to. Sometimes the reason a game is loved by a reviewer is the same reason that I'd avoid a game. And I'd say the review has been informative and useful to me.

Why do you read game reviews? I used to read them to learn about new games that I may be interested to try or to buy. Sometimes I read them to learn more about games I am already interested in. Nowadays I find that I tend to skim them more than I read them. I am buying fewer games than before. I probably get more enjoyment out of reading entertaining gaming anecdotes (which are more fun if they are related to games I already know) than reading information that helps with buying decisions.

To simply measure a game on a good to bad scale, regardless of the number of points on that scale, misses one important consideration - the context of the game being played. E.g. the audience. If you play games with different groups, some games will work with one group but not another. Their gaming tastes, how "hardcore" they are, their ages, their professions, all do matter. I imagine many games will have spectrums of different widths and depths if we try to measure how well they work under different circumstances. Other than people, there is also time of day, energy level of the group, weather, and festive seasons (gambling games for Chinese New Year? Horror games for Halloween?) to consider. Sometimes I ask myself - what game am I in the mood for today? It certainly helps to have a decent collection of games. No matter what kind of mood I am in, as long as it is a mood for playing games, I can always pull something off the shelf that is right for that moment. I guess that's partly why my buying urge has cooled much lately.

Perhaps there are no bad games. There are only bad moments to play a particular game.

Take 6 / Category 5 / 6 Nimmt would be funny as a drinking game with a rowdy group.


6 comments:

Luke Eastlake said...

I have to agree with you on the BGG rating being a hinderance at times. My group originally refused to play anything outside of the BGG top 100 when we first started playing.

However, recently my persistance has paid off and we've managed to get a few 'lower' rated games to the table. On Saturday we got a game played with a rating of a 6.3 (around 1000 on the list) and it was one of the best game nights we've had in a long time.

Then again, I too fall for the BGG rankings and it definitely influences what games I am likely to purchase. The only thing that tends to change my opinion of 'lower' rated games is price.

trainingagamer said...

I actually find the BGG ratings useful, but a serious pinch of salt needs to be applied. Because of the way the ratings are calculated, I figure that any rating above 6 is actually pretty good. After all, there are currently only 6 games with ratings above 8, and a little more than 250 above 7. Any game that has a score above 6 clearly has a good number of people that like it, and it is at least worthy of consideration.

When it comes to more child-friendly games, I feel a rating above 5 is reasonable, as many BGGers don't want to play them, so any higher and there must really be something to recommend them.

But the ratings are just an indicator. I wouldn't generally buy a game just because it has a high rating; I want to know how the game plays, what the theme is like, etc. and also the price is a factor (highly rated and costs $10? I'm willing to try that!).

As for personal ratings on a review, I must admit I quite like them, though a clear, qualitative scheme (like you suggested, something like "not for me" to "love it") works really well as an overall summary. But again, with reviews, you really need to know how your opinions match with those of the reviewer, so you can tell whether you are likely to agree with them or not.

Thanks for the post, though. Very thoughtful and interesting.

Jason said...

Yeah, lots of different thoughts about reviews. I actually pay no attention to the BGG ratings. And because of that, I like to read in-depth reviews with a rules overview (not bullet point by bullet point rehash, but enough to give me a good idea about how the games plays). That's how I do my "research" into if I think I - and my boys - will like a game before I buy.

Frank Feldmann said...

Great article. I think the ratings on BGG may also be skewed towards new games, since the BGG population has exploded over the past couple of years. Newer gamers may not be as familiar with older games.

The driving forces for me are game length and complexity, which aren't directly reflected in ratings.

As for reviews, I completely agree that I want an idea of the whole gaming experience, which may take several plays. Audience is so important, as you say. When I review, I try to examine the experience from the perspective of the family or casual gamer, and that can take an otherwise great game and make it not so great.

Hiew Chok Sien 邱卓成 said...

Good point. Indeed BGG ratings are skewed towards newer games because of how the BGG population has been growing.

One thing that I know I will not be able to do is review a game after playing it at least 3 times. If I try to follow this rule, I will have very little to say at my blog. :-) I often hesitate to call my blog posts "reviews". "First impressions" is more accurate. So I preface my posts with the number of times I've played a game, and the number of players.

Frank Feldmann said...

I understand that time pressure can mean a review only after one play. Honestly, there are games that I have played only once that I know are great or I know are awful. Part of the trick is acknowledging that might be a better game with different people, as we said.

I guess I am lucky. On my blog (http://sofrankly.com), I am targeting family and casual players, which means that many games fail to make the cut just do to their complexity. I have enough experience as a gamer and a parent to know that some games will not work.