Some Thoughts on Beer Ratings and Reviews

Over the years I’ve dabbled a bit in homebrewing. Right after college my roommate and I bought a starter kit and tried to make a few batches of beer, the results varied from decent to so bad that a group of guys in their early 20’s dumped it out. My buddy went to medical school, I went to graduate school and we gave away the equipment, neither of us had the time to keep up the hobby. When I finished my PhD my girlfriend (who is now my wife) got me a new homebrew kit and I started up the hobby again with much better results, but over the last few years I’ve completely stopped brewing. There are a number of reasons for this, I have two young kids now so my free time is non-existent and I’ve cut way back on my drinking, so 2 cases of homebrew would take forever to finish (especially if I am also drinking beers to review on Hoppy Boston). Probably the biggest reason I’ve stopped homebrewing is that my palate has refined over the last 5+ years of carefully critiquing beers, and I’ve realized that the majority of the beers I brewed were mediocre to good, but nothing took the next step to very good or great, so it was hard to justify drinking it over a stellar commercial beer.

Hoppy Boston Hopsters Belgian IPA

I think the biggest thing I learned from my years of homebrewing is that it’s tough to make a great beer and that the brewers who do it on a consistent basis are very talented. This is why the current state of keyboard warrior crowd-sourced beer reviews is infuriating to me. I’m not talking about the average beer geek who keeps a general scale in their head and uses it to keep track of which beers they loved and which missed the mark. That is totally fine. I mean the people who use crowd-sourced apps to trash any beers that didn’t require an online trade or a wait in line as “shelfies” or “drain pours” or “meh”. It is this behavior that skews the ratings on crowd-sourced site in favor of a couple styles and a handful of breweries to such an extent that the ratings themselves are nearly useless. Unfortunately, a substantial subset of craft beer drinkers, especially people who are new to craft beer, take these ratings extremely seriously. Ask any bottle shop employee for stories about dealing with customers who are shopping with their  Untappd app open and be prepared to shake your head in disgust.

Tree House Line

Here is a secret that will shock many of the people who wait in line for hours every weekend to buy whalez: the vast majority of the craft beer that readily is available in a quality bottle shop is good to great. Breweries that reach the point where they can distribute usually know what they are doing, and often make much more consistent beer compared to some smaller places that are constantly shuffling recipes. Obviously, there are exceptions and freshness is extremely important with many styles. There are also breweries that don’t know what they are doing and make some pretty bad beer, but they are in the minority. So if you are constantly trashing beers on Untappd either you have terrible luck picking beers, you shop at stores that are selling old beer and you don’t check the dates, or you have no idea what you are talking about. I’m guessing the majority of the Untappd accounts that are loaded with reviews under 3 stars are the last option on the list.

The other issue with crowd-sourced reviews is that a few styles get completely over-rated compared to the rest. 38 of the top 50 beers on Beer Advocate are IPAs or imperial stouts. on Untappd it is 43 out of 50 in those two styles. That is completely insane, there are world-class beers being made in every potential style, but if the beer isn’t loaded with hops, booze or both a rather large segment of the crowd on these sites can’t appreciate it. Unfortunately, this drives sales and breweries that are just scraping by making traditional lagers or Belgian styles feel compelled to make hop bombs and barrel aged stouts to stay relevant. I think these crowd-sourced rating sites and the people who read them like gospel are a gigantic negative for craft beer, stifling true creativity and innovation.

When I started Hoppy Boston I struggled on how I would do my beer reviews. I always wanted beer reviews to be a key part of the site, the scene was exploding and there were so many options available, I wanted Hoppy Boston to be a place readers could go for recommendations of beers they should try. I decided on a 5 point scale, I felt that it was important to have some kind of numbering system to distinguish between good, very good and great beers. Some days I wish I did a 10 or even 100 point scale, there could be a lot more nuance. Other days I wish I didn’t have a scale at all, I just wrote about the beers, but I don’t think that system would work very well, it is hard to really sell the difference between a very good and a world class beer without some sort of numerical scoring.

I get somewhat regular comments about the fact that most of my scores are high, essentially every beer I review is between 3.5 and 5 out of 5 stars, and most are 4.0 or better. When I started writing reviews I would review whatever beers I grabbed, and if I didn’t like them I would write that. In the first year, there were a few cases where I wrote things I regret now, the beers were clearly out of date or had a packaging issue and I was too inexperienced to realize this at the time. I haven’t deleted anything from Hoppy Boston, but I am tempted on occasion. On a side note, some of the writing was abysmal in the early days too, it took a while and some help from my wife (who was my editor for a while before the kids were born) to really hit my stride.

To be honest, I also hated writing negative reviews. I know some people take perverse pleasure trashing the hard work of others (just read the comments on any crowd-sourced website), but that isn’t me. To be completely honest, it is rarely warranted. Now I buy plenty of beers with the intention of writing reviews and I only write up beers that are my favorites in the batch or decent/good beers from a brewery I want to highlight a little more. Even the beers I don’t write up are rarely bad, some don’t fit my personal preferences for a style or are just unremarkable, occasionally some are just too weird or experimental, but the vast majority are pretty good or better.

Smuttynose IPA

I’ve heard criticism that any site that doesn’t publish negative reviews isn’t “being real” or isn’t helpful. I actually find positive reviews to be way more helpful when I choose which beers I want to try. If I walk into a good bottle shop there will be hundreds of beers I’ve never tried, even if I just limit myself to beers from New England breweries. I could staff Hoppy Boston with a team of reviewers each churning out multiple articles a week (which would be a huge feat for a blog that is a hobby and makes no revenue) and we wouldn’t scratch the surface of covering all of the beers available at any time in the region. With that in mind, which type of review is more helpful, one recommending a beer you should try or one trashing just one of the thousands of available beers.

I think that beer reviews and ratings can be very useful. Some beer writers look down of beer reviews, and while I agree that there are many other interesting parts of the industry to write about, I also like reading about individual beers. I think there can be a balance, where people have fun on crowd-sourced apps without taking the scores too seriously and people can read reviews on sites like Hoppy Boston are take them as fun recommendations while feeling free to disagree. I am sure there are beers I’ve scored high that some of you don’t particularly care for, and beers you love that I thought were just OK. That is fine, we all have different preferences. I just encourage everyone to go out, try some beers in a range of styles from different breweries and form your own opinions. You might discover that a beer with a mediocre Untappd score sitting on a bottle shop shelf is actually your new favorite!

With that in mind, what are your opinions on crowdsourced beer ratings? What about written beer reviews? What do you find helpful when you are trying to select new beers to try or breweries to visit?

3 Comments

  1. Beer rating are guides. A guide isn’t gospel. Too few people understand that. It reminds me of when I worked in the car industry and buyers would act like Consumer Reports or KBB was written by Moses.

  2. Couldn’t agree with you more. I rarely have a beer that is really bad. I used to think I was not discriminating enough because I liked so many. Now I just describe the beer as I experience it and try to disclose any bias I have toward a style.
    Also agree with you on overrating certain styles. Ayinger Bavarian Pilsner is a world class German Pilsner; but it has a 3.54 rating on Untappd! Stone IPA is borderline undrinkable for me, yet it has a 3.8 rating. Those two example show me how useless these ratings are.

  3. I find Untappd wildly useful. I use it to track beers I’ve had and how I felt they were rated at the time I drink them. I only really drink styles I like and I know I’m a lenient grader so my rating curve definitely skews higher. But I know that and it makes my experience deciding on what new beer to try easier.

    To the person complaining above about Stone IPA being a 3.8 and undrinkable….uhh yeah that’s a pretty bad rating for an IPA on Untappd. I think Untappd should normalize their rankings by style so there’s a consistent baseline and the lists are just dominated by IPAs/Stouts.

Leave a comment