Barley Mowat 

Archive for the ‘jantweats’ tag

The Problem With Beer Ratings

with 5 comments

As some of you might have noticed, I’ve started putting up beer reviews. I’ve avoided doing just this very thing for a long time because, much as with Most Things In General, I have a problem with beer ratings.

It all boils down to this main issue: people are not objective. We spend a lot of energy prancing around the issue and claiming to solve the problem through various ingenious inventions, but ultimately people suck at rating things.

If I were to take a bucket of my very own homebrew swill and pour it into three bottles, marked as “OK Spring”, “Driftwood” and “Chuck’s Discount Swillery” then ask you to rate all three, people would rate each very different.

Like it or not, your impression of the label sets an expectation in your mind and you adjust your review accordingly. It’s not just brand-blindness, beer from a bottle labeled “Driftwood” actually, truthfully, tastes better to you than the same beer sopped up with a mop and wrung out into the glass in front of you (it’s a clean mop… made of hops).

Okay, so let’s remove the labels; this is where it gets interesting. You see, it doesn’t really matter WHAT is in the bottles. You’re going to give the beer a 7/10, or a 4/5, or a 20/25. Sure, maybe not every time, but overall that will be your average, despite the very real fact that “average” should trend towards 50%. As well, statistically you won’t deviate very much, and when you do deviate, your mood at the time of rating will have a much, much higher effect on the result than the actual beer.

Ironically beer salted with tears is actually quite delicious, but no one has realized it yet

Don’t believe me? Take a look at the recent ratings from two professional reviewers, Joe Wiebe (@thirstywriter) and Jan Zeschky (@jantweats). Sure looks like a lot of 4/5 and 20/25. Whoa, all the way down to 3.5/5? That beer must have sucked. 21.5/25? Best Beer Ever.

I don’t want to take anything away from Joe and Jan. They’re rating beers for publications that demand a simple number, and rating beer is freaking hard. Plus, it’s hard to balance out not wanting to rate too highly (and lose credibility with people who don’t like that beer) with not wanting to rate too lowly (and risk the publication not running the review out of fear of insulting advertisers). In their shoes I’d likely have the exact same results. It’s not the people that are at fault, it’s our desire to put a number on things.

It isn’t just beer, either. The wine world has been struggling with this problem for a while, too. Ever seen a wine rated less than 80/100? Me neither. This is so prevalent that now some reviewers have switched to rating wine out of 20… only to see their average reviews move up to compensate.

It’s almost as if we don’t want to use the lower end of the scale because we’re afraid of offending the beer (or wine, or restaurant). However, the reality of the matter is that there ARE shitty wines and beers out there, and there ARE products deserving of 1/5, 1/10, 1/25 or even 1/100.

Or even 1/10030, as the case may be.

This is why I like RateBeer for beer ratings, as imperfect as it is. It lets all the humans go fuss about whether a beer is 3.80 or 3.85 out of 5 (or maybe even 4.05), and then it applies cold hard statistics to rank all beers based on percentile. That horrible horrible beer we inexplicably gave 2.5/5.0 to? RateBeer boils that down to a much more honest 15/100.

And now to bring all this back to me and my beer reviews, so I can wrap this up and go have a Friday pint. As part of my reviews, you might notice there are no ratings. RateBeer has that covered; you want a single number to impossibly summarize the complexity that is a particular beer? Go there.

Do you want my opinion on what’s going on with this beer? Read my review. I will slip in a Barley Mowat Seal of Approval, but that is not a rating. That’s just how excited I, personally, am about this beer. Bronze is “yeah, I’ll buy it again if I see it.” Silver is “I will go out of my way to purchase this” and Gold is “If you are holding the last bottle of this, I will slit your throat and take it. And then kick you in the jewels, because that’s just the kind of guy I am.”

The vast majority of beers, though, get nothing, not because they’re awful (although some truly are), but rather because they don’t excite me… no, not like that… okay, maybe a little like that.

Written by chuck

November 2nd, 2012 at 3:42 pm

Posted in Beer and You

Tagged with ,