Whatever flaws the process might have, however, I absolutely reject any insinuations about judge bias, dirty tricks or corruption. The judges were blinded and while we could recognize some of the beers we judged, many of us were surprised by the beers we ended up choosing. It’s not as though any single brewery made a surprise sweep of the awards.

At the end of the day, this contest is too unimportant for anyone to bother trying to corrupt it. It really is just a bunch of beer geeks gathered around arguing about which beer is best.

I ended up judging because someone had a stroke. I’m pretty sure the stroke wasn’t caused by agents of a sinister craft brewery trying to influence the results.

Dave, your insights into the judging process are worthy of their own post. Next time I suggest employing fisticuffs to resolve any disputes, aprons vs gloves meets beer tasting.

I would really like to see a list of all the beers entered into each category. I’d also like to see a list of all the judges. If Ben Coli was a last minute replacement, I should be able to get in by “delaying” a few of the judges next year.

I also think Brewery Creek should just enter every beer BC beer. I’d like to be sure Dead Frog’s collection of lagers didn’t win because they’re terrible, not because they didn’t enter.

The style guidelines vary in their accuracy and relevence to actual beers, particularly European styles. Some of these guidelines have been completely plucked from thin air and seem to have arbitrarily been made up. (I quote “Robust and Brown Porter” being a classic nonsense. it is a much repeated so-called style that has no basis in history) see Martyn Cornell’s book “Amber Gold and Black” for a British beer and brewing historian’s description of this style and its evolution. -(Don’t get me started on ‘English’ IPAs with beers winning those categories that have no English hops in them)

The other problem with “brewing to style” is that it does not take into consideration the evolution of beer and the changes to it. A so called “classic” Porter is what? A porter with the grain bill from the early 1700s when this beer was first developed and mentioned or from the 1920s when it was beginning its death throws in the UK? different grain bills alcohol and therefore flavour. both are historic but which is “Classic”?

My information from a Judge who will remain nameless is that the crazy discussions after the beers were tasted ended up with whoever had the loudest voice tended to get their way (not my imformant BTW!)

Why weren’t the marks just tallied, so that no personalities came into play? Each judge is as important as the other. this tends to be how wine and spirits are judged. There is some discussion if two have very close marks for the top spot. Usually they are re-evaluated blind to double check the results.

Also sponsors should not have anything to do with the judging process. This is a pretty basic rule to ensure fair play. Just look at the Diageo / Brewdog debacle in the UK a few months ago: http://www.telegraph.co.uk/finance/newsbysector/retailandconsumer/9255369/Scottish-beer-company-BrewDog-forces-Diageo-to-apologise-over-dirty-tricks-at-awards.html

Anyway the BJCP way is not the only or best way and this homebrewing judgeing program cannot be where a qualatative assessment of beer in professional awards. There are beverage industry standards that are long established, let’s not try to re-invent the wheel

1) How do you empirically say “these judges are qualified”? BJCP qualification is an attempt to do that. Otherwise you run the risk of putting together a room of people who may or may not even be able to tell you whether a beer is malt- or hop-forward or not, let alone whether it’s flawed and how.
2) If you use BJCP as your guideline, you do tend to initially measure a beer up to how closely it matches the style. Some are obviously flawed and worth rejecting outright, others may not be a perfect match though they have something intangible about them that keep them on your list. The goal of that part of judging is to, as objectively as possible, find the standouts from a flight.
3) The standouts then advance to a mini-best of show round within their category. There could be 3, there could be 10, it depends on how many beers were entered. Within that best of show you talk through the merits and weaknesses of each one, and use a combination of objective and subjective criteria to determine which should be eliminated until you’re down to three. Then you choose which order the three should be placed. Different judges will have different criteria at this part of the process, some will vote up most-to-style beers while others will vote up beers that they personally enjoy more. Neither of those are really wrong, and if you get a diverse enough set of talented judges this should balance out.
4) Throughout all this, you do your best to not let recognizing a beer bias your judgement. You would be surprised — no, shocked — to find out some of the beers I recognized but consciously eliminated/advanced due to weaknesses/merit. You’re judging the beer that was poured, not previous times when you know it has been far better/worse. If it doesn’t stand out in the flight, it should rightfully be eliminated. (And the reverse if it does.) Sometimes this can explain why a beer that has won in past years didn’t win this year.
5) Yep, in BC we just don’t find a lot of interpretations of some styles. In some cases we don’t have enough of a certain style to even round out a larger parent category. (See last year’s grouping of sour beers with fruit beers. At least we’re past that this year.) Your options: automatically give a first place medal to someone just for showing up, don’t give any medals in those categories and ignore the beers that were entered, or use some discretion to put related beers together and try to judge each on its own merits. None are ideal, but I think the last option is the best you can do in that circumstance. The BJCP specialty beer category is the established model we’re following here; you can’t really compare a winter warmer to a Belgian IPA to a Sticke altbier, but they provide guidance for how to do it anyway and that’s the way it works in competitions: http://www.bjcp.org/2008styles/style23.php
6) The results depend on the luck of the pour as well as skill of the judges. Judging early in the day when your palate is fresh is easier, judging later in the day when you’ve been through multiple flights gets a bit harder. Organizers balance that by starting with the more subtle beers and working up to the more flavourful ones, but it’s still a factor. Breaking up the day into multi-day judging sprints would be silly, the only alternative here is having more qualified judges there. We’re getting more all the time, should be easier in a few years.
7) “how did Y win when X is clearly a superior take on the style?” — fair question, except if X didn’t enter, it can’t possibly win. Not all breweries enter. Also, if a great beer is entered in the wrong category, it’s probably not going to win. This happens more often than you’d think. It’s not always the brewers themselves who are the ones handling entries, it could be a marketing person. Think a marketing person is likely to nail beer categories?
8) I didn’t agree with all the outcomes either. Many were baffling. I remember advocating for some beers that I thought should have been placed higher up, and being outvoted by other very qualified judges. Hey, that’s taste for you, everyone has different preferences. Who’s right, them or me? Some people would have agreed with me, some people will agree with them. Even with BJCP as the yardstick, it gets subjective. As well it should. This is beer, not math.

Maybe the biggest problem is that some of these behind-the-scenes factors aren’t exactly public knowledge, so people are free to look at the outcomes and speculate about how the results were arrived and assume that was flawed. Maybe this comment will help a bit.

Personally, I’d like to see a full list of all breweries who entered each year so that at least the “why didn’t X win?” question is cleared up. This seems like low-hanging fruit that the organizers should be able to easily address.

For another judge’s take, see Ben Coli’s post: http://sloppygourmand.com/2012/10/14/2012-bc-beer-award-winners/