How Biased Are Pipe Tobacco Reviews

[ClinchKnot]

I don’t know if tobacco reviews are biased, but I suspect that if someone known to do a lot of thorough reviews suddenly started raving about a particular blend or blender while everyone else was saying the tobacco sucked the game would be over fairly quickly.

I don’t use tobacco review ‘stars’ to determine if I try something or not. Those little three or four star scores just aren’t developed enough to be any kind of reliable predictor of whether or not I will like a blend.

What I do with tobacco reviews is read them all while I am smoking a new blend. I will do that for two or three bowls, and I will take notes. Usually what I pick up in a new blend is scattershot over the reviews. One person will mention spiciness of a certain kind and I get that. One person may mention citrus or sourness and I may recognize that and write it down. I’m never going to be sommelier of tobaccos, don’t have a superhuman palate, but the reviews do help me figure out what I am picking up and enjoy (or dislike) about a blend and that serves a very useful purpose to me.

 

[BPatrick]

I don’t know if tobacco reviews are biased, but I suspect that if someone known to do a lot of thorough reviews suddenly started raving about a particular blend or blender while everyone else was saying the tobacco sucked the game would be over fairly quickly.

I don’t use tobacco review ‘stars’ to determine if I try something or not. Those little three or four star scores just aren’t developed enough to be any kind of reliable predictor of whether or not I will like a blend.

What I do with tobacco reviews is read them all while I am smoking a new blend. I will do that for two or three bowls, and I will take notes. Usually what I pick up in a new blend is scattershot over the reviews. One person will mention spiciness of a certain kind and I get that. One person may mention citrus or sourness and I may recognize that and write it down. I’m never going to be sommelier of tobaccos, don’t have a superhuman palate, but the reviews do help me figure out what I am picking up and enjoy (or dislike) about a blend and that serves a very useful purpose to me.

I watch several as entertainment as I enjoy well made pipe content and good presentation. I learn a lot and make a decision to pull the trigger or not on a blend. That's a great idea to smoke several bowls of the blend while watching reviews to see if you can pick out things. Gonna have to try that.

 

[Sgetz]

For me it's quite simple. I can try all blends so I have to source informative opinion somewhere. I don't experience taste like some of the master reviewers but I've found a few who like what I like so off I go!

 

[popeofpiping]

Tobacco reviews are just diary entries.

 

[krizzose]

Interesting. I only recall his reviews commenting on the components not the proportions.

I hope it was obvious I didn’t mean that he produced a breakdown by percentages. I doubt he’s privy to that info in most cases. Rather he gives his best sense of the components’ relative proportions. Of course, that is still his subjective opinion, but far more objectively useful for me than “this tastes good”. YMMV.

For example, in his review of GLP Windjammer (chosen as I’m currently smoking it), he uses the following statements:

“…. They [the Virginias] are the lead components….”

“….The perique is an important supporting player….”

“….They [the Burleys] are a notch or two above the condimental slot…”

“….The toasted unflavored black cavendish … as a condiment….”

Call me crazy, but that helps me get a sense of how much of each component is in the blend

 

[PaulDM]

It's not the bias so much as the specificity and detail that throws me off. Perhaps some people have super palates and mine is useless, but I can't relate. I prefer more general impressions and simple reviews rather than a detailed analysis and breakdown of tobacco type in the blend. But to each his own,

 

[bpinkstaff]

I did a scientific study of this, and it determined i like what i like no matter what everyone else likes! Ok there was no science involved lol!

 

[BPatrick]

I hope it was obvious I didn’t mean that he produced a breakdown by percentages. I doubt he’s privy to that info in most cases. Rather he gives his best sense of the components’ relative proportions. Of course, that is still his subjective opinion, but far more objectively useful for me than “this tastes good”. YMMV.

For example, in his review of GLP Windjammer (chosen as I’m currently smoking it), he uses the following statements:

“…. They [the Virginias] are the lead components….”

“….The perique is an important supporting player….”

“….They [the Burleys] are a notch or two above the condimental slot…”

“….The toasted unflavored black cavendish … as a condiment….”

Call me crazy, but that helps me get a sense of how much of each component is in the blend

Agreed....that's a very helpful breakdown.

 

[BPatrick]

I did a scientific study of this, and it determined i like what i like no matter what everyone else likes! Ok there was no science involved lol!

Cause...Science!

 

[FLDRD]

I once smoked a 3 star blend - and I liked it!

 

[RudyH]

The only reliable way of finding new tobaccos that I like is to actually try some. I end up throwing some away and cellaring others in hopes that they will age well. When I get lucky, I buy more. My favorite tobaccos smoke better in some pipes than in others. The specific pipe matters a lot.

 

[telescopes]

Reviews are very helpful. They help not with just explaining the components, but give insight into nic hit, individual tobacco levels in terms of supporting and lead flavors, and a good idea of whether or not the mix seems to be working as intended.

 

[bullet08]

Reviews give general idea as to what that blend will be like. But you'll have to make the final decision as to buy and smoke it or not. Never really had tobacco bad enough to throw away. Doesn't mean I'll smoke it, but I don't like throwing things away. Maybe 50 years down the line, some pipe smoker will find it and wonder if he should dehydrate it.

 

[sablebrush52]

The only reliable way of finding new tobaccos that I like is to actually try some.

Bingo! Reviews are useful to a limited extent. Written descriptions are at best an an approximation, no matter how detailed, for describing sensation.
Just as one picture is worth a thousand words, so is one taste, one sniff, one touch, etc.

 

[khiddy]

Reviews are very helpful. They help not with just explaining the components, but give insight into nic hit, individual tobacco levels in terms of supporting and lead flavors, and a good idea of whether or not the mix seems to be working as intended.

I dunno about how helpful they are about nicotine levels. I was reading through the reviews of some Captain Earle blends yesterday, and on back-to-back reviews there was a claim that a particular blend was high in nicotine, while the next said it was a mild hit.

The same for claims of “consistency” - I see reviewers that I recognize and respect who will say opposite things about the same blend’s consistency, and I’m left with no more info than if I’d never read the reviews at all.

I know it comes down to following the “preponderance of evidence” standard, but these conflicts appear all the time in reviews.

 

[cosmicfolklore]

I don't read reviews to see if anyone else liked it, nor what exactly it is going to taste like. I tend to just look at Jim's reviews on there, or to read the weird ones for entertainment. It really amazes me that folks would take reviews so seriously. I have more a heightened sense of taste than most, except maybe Jim... but, even with his reviews, I don't always taste the tobacco the same way he did.

It has actually been a few years since I was on that site. But, you'll get better feedback on what you might like by just making friends on here and discussing tobaccos with folks. Sure, it takes time, but you'll be much more satisfied with the tobaccos you buy this way, as opposed to just looking to see what is most popular. In many instances, I have found popularity doesn't taste very good.

 

[telescopes]

I know it comes down to following the “preponderance of evidence” standard, but these conflicts appear all the time in reviews.

Like with most data analysis, correlation is sometimes all we can best hope to find. But, it's a start and if used to help the reader ask more questions about a particular blend, the reader can then do additional research into the blend in question. And isn't that what it always takes. Reviews are a good place to begin. But as we all agree, they are not the final step. That said, not using them as a tool isn't efficient or helpful. They are what they are, but when all is said and done, why would one not consider them to be at least useful for beginning one's research.

 

[Uguccione]

I can no longer connect to "Tobacco Review" because the government (or whoever) has blocked internet access to all foreign tobacconists, including some sites that talk about tobacco (sooner or later it will be Pipes Magazine's turn, who knows.... if one day you will no longer see me... :D ).
On TR I read a few reviews here and there, being careful to view them from the most recent and not the most popular. There is no point in reading a review from 15 years ago, in the meantime the tobacco could have changed who knows how many times. It is also important to check how many years of aging the tobacco had at the time of the review.
That said, I found the average ratings on the strength of the tobacco, the room note, etc. useful...
If you pay attention, those are always quite reliable as they are the result of an average.

My ideal review would be one that tells me about the character of a tobacco, its personality; that the reviewer tells me what he felt while smoking that tobacco, what sensations it aroused in him. It would still be a subjective thing - okay - but I would find it more useful than a list of flavors that you feel and I don't.

It's like describing a person by saying that he is tall, blond, blue eyes... You haven't told me anything about him. Tell me instead if you found him nice, honest, sincere...

 

[sardonicus87]

I only skimmed, but star ratings might be more off than what people say. Reviews are subjective obviously, but star ratings are too, and this goes for everything.

Some people use star level to reflect their own enjoyment.

Others use it more objectively (as much as possible) in, how well the blend achieves what it is trying to do, regardless of how much they do or don't like it, relative to other blends in the same genre. For instance, trucks generally are meant to haul stuff as opposed to cars, there may be a fancy looking truck that rides smooth, but it has next to no cargo space and is crap at hauling stuf. One guy likes the truck but he uses it like he would a car and rates it 5/5 Another likes how the truck rides and looks, but rates it 2/5 because it can't do basic truck stuff, like carry grandma's sofa.

I forget what the rating system is called, but there's a similar one for rating art. It's less about whether it is good and more about, is the selected media utilized properly, does it fit the theme of the message, is the message discernable and clear, etc. Basically a focus on the technical execution rather than enjoyment.

Never mind that you have people giving low star ratings because of a shipping problem (rate the shipper for that, not the product), or because they're mad they didn't get to get any, etc, all the stuff talked about already.

But then you have other problems with rating systems. Some people will rate higher or lower than they feel a blend deserves because they feel the rating it currently has is lower or higher than it should be and they're trying to offset and shift the overall rating closer to what they feel it should be. Others might rate higher or lower than they otherwise to preempt bad reviews (bad being overly negative OR overly positive) they think will happen. And further still, others may have an idea of what they would give it, but give in a different direction to either be contrarian or to toe the line (and sometimes this is on purpose, other times it's unconscious cognitive bias). Or they rate based on what they think is the "right answer" or what others want to hear. And still there's further motivations for purposely false ratings and reasons for unconscious shifts in ratings.

That's all without considering the fact that many people only rate things very high or very low because they only go out of their way to rate something when they're highly upset or extremely pleased for some reason or other. Look at any product or movie or service with thousands of ratings and there's three basic patterns... extremely high, extremely low, and the middle where when you look at the mid-rated, it tends to be a U shape with few reviews near it's average and many reviews up and down at the extremes.

You can't even trust professional critics. There's plenty of movies out there for example that got panned by critics and loved by audiences and vice verse.

But it's not all subjectivity or all bias or all objective or even an even mix of the three, and not all bias is conscious and/or malicious.