“Only about 10% of [US wine] judges are consistent and those judges who were consistent one year were ordinary the next year….Chance has a great deal to do with the awards that wines win.”
so many feathers
I was also struck by the sympathetic tone of contributors to more professional forums such as Linkedin’s Wine Business Network
Wine tasting may not be a science, but I’ll break ranks with many wine writers by saying that I agree with the basic theme of the criticisms: wine assessment and the way wine is written about do lay themselves wide open to accusations of being pseudo-scientific.
Let’s start with so-called Parker Points. Giving a wine 89 or 90 or 91 points implies considerable precision, especially when, as we know, the side of the 90-point line on which a wine sits can have a huge impact on its marketability and the price it can command. (17.5/20 implies similar precision – and actually involves it if one considers that users of the 100-point scale rarely dip below 79 and those who prefer rating out of 20 treat 10 as their floor).
To the annoyance of many in the wine world, points have been welcomed both by consumers who find them an invaluable route through the wine jungle, and by producers and distributors who use them to drive sales.
But, as I’ve often been asked by consumers, how does a critic arrive at a mark of 89 rather than 90, say? The OIV competitions attempt to give structure to their scoring process by requiring judges to indicate the number of points they have allocated for colour, nose, typicity and so on – but they still allow a margin for tasters to show gut-preference. Besides, like many other OIV tasters, I’ll admit to coming up with the mark first and filling the category numbers afterwards – just as most critics do when giving their “Parker” point.
However they arrive at their mark, in theory at least, trained palates should achieve some level of consistency – and they often do. But it’s far from absolute. Robert Parker admitted (to the author of the WSJ piece) that his marks for the same wine can deviate by two or three points, and it’s a rare taster who’d stake anything of value on greater consistency than that. But three points on a scale that runs from 70-100 sounds very much like an error-rate of 10%. And when the variable is applied to wines getting 85-100 (which is likely to be the case), it’s a lot more significant.
As some of the people who responded to the Observer article pointed out, there is a long list of reasonable explanations why even the most skilled and experienced taster might give – possibly widely – different marks to the same wine. Let’s consider just a few of them:
- The place and time. A wine may unsurprisingly seem very different when sampled in the producer’s cellar to the way it tastes in a blind line-up in a competition. A few weeks can also have an effect on a wine’s evolution, especially when it is young. (Think of all the variations between en primeur tasting notes written in early April and when the same wines are presented in June at Vinexpo.)
- The time of day. A wine tasted at 9am, shortly after breakfast might be rated differently by a hungry taster at 1pm, or by one who has just enjoyed lunch at 2.30.
- Physical state. The taster’s state of tiredness, stress or physical health. Zinc deficiency, for example, can lead to a condition called Dysgeusia – “a distortion of the sense of taste” – often associated with complaints of foods having a a metallic taste. Patients undergoing chemotherapy frequently complain of their palates going awry. This Japanese piece of research reveals that physical exhaustion lowers the sucrose-perception threshold. The researchers on that occasion worked with athletes who’d run a half marathon, but some of the same effects might well be found in a taster who’s had a sleepless night dealing with a young baby, for example. Another – US – study found that stress increases the perception of bitterness. Further causes of “bitter-taste-in-mout” include, acid reflux disease or GERD, Hiatal Hernias (or other hernias), tooth decay or gum disease, a condition that affects the oesophagus), bacteria that may infect around two-thirds of the people in the world), Oral Cancers, Aspiration Pneumonia and syphilis.
- Mood. Happiness, sadness, fear and anger can affect the way we perceive taste. According to a Los Angeles Times article,
Given a neutral-tasting shot of diluted blue Gatorade, participants in a study in press at the journal PLoS One thought the beverage tasted more delicious after reading about someone being morally virtuous and more disgusting after reading about a moral transgression.
- External distractions
- The position in a line-up
- The temperature of the wine
- The length of time the bottle has been open
- The glass from which it is tasted (if you are to believe Georg Riedel’s blindfold demonstrations)
- Shipping. The old notion that some wines “don’t travel” may have some validity. (New Zealand wine authority Bob Campbell addressed this issue last yearDr Neill McCallum, founder of Dry River Wines in Martinborough, New Zealand. McCallum, who gained his PhD at Oxford studying the relevant branch of chemistry, theorised that movement through travel breaks the hydrogen ions in wine resulting in muted flavour and aroma. Hydrogen ions do re-form, although not completely, according to McCallum, resulting in partial recovery of aroma and flavour when wine has a chance to recover after travel. He also notes that damage to hydrogen ions is significantly lower if the wine is cooler, supporting the case for shipping wine in refrigerated containers.
- Lunar cycles. If you believe in biodynamics, the calendar is divided between four different kinds of day, depending on the lunar cycle.
- Altitude. It is acknowledged that foods and drinks taste different in airplanes, but that might be because of the lack of moisture in the cabin air: wines in particular seem to taste more tannic and more alcoholic. As a 2010 German study found,
But the altitude itself may have a role to play. Another study – published by Maga JA, Lorenz K.in the US National Library of Medicine National Institutes of Health – suggests that from 5-10,000 feet, people become more tolerant to bitterness.
- Barometric pressure. If altitude affects taste, so, logically, might the weather on the day the wine is tasted.
- Humidity – or lack of it. The dry conditions in airline cabins are widely blamed for numbing tastebuds.
As usual, I pre-tasted all of the bottles for cork taint – and found a couple that showed clear signs of TCA. Two litres of sour milk out of 80 or so bottle, or a couple of bad eggs in a dozen cartons would be a cause for concern, but in the crazy world of wine, the sommelier at the venue and I shrugged off this level of faultiness in highly costly reds and whites, as par for the course. Far more concerning to me, however, were the more subtle variations I encountered in the other wines. When tasted really carefully – something wines of this calibre deserved – there was only one set that were truly 100% consistent: the Gajas. All the others included at least one or two bottles that tasted different enough from the others to warrant the addition or removal at least a point or from their score.
Now – and this is important – I’m not claiming any exceptional tasting skills. Far from it. In fact, I tested my belief in the irregularity of the bottles by setting up a few three-glass tastings for the sommelier and an enthusiastic waiter. Even without any experience of the wines we were looking at, in three of the four instances, they had little difficulty in spotting the odd man out.
All the bottles were sealed with natural corks and I suspect that the poorer examples almost certainly suffered from very low level TCA or random oxidation – both of which flatten flavour, It is harder to explain the outstandingly better bottles. I suspect that particularly “good” natural corks may have provided optimum oxygen ingress plus possibly, that rarely discussed factor: “gout de bouchon” – the subtle flavour of non-tainted cork oak.
This naturally leads me on to one of the anomalies of wine competitions. I have chaired or co-chaired over 50 International Wine Challenges and other similar events across the globe and judged at dozens more. One of the things all these contests have in common is the encouragement to tasters to call for a second bottle if they have doubts about the first. On occasion, a third bottle can be requested.
Sometimes the call is easy: a wine reeks of TCA, vinegar or sherry. All too often, however, it’s more marginal. Tasters discuss whether it’s worth pulling another cork (or unscrewing a cap). Sometimes the second bottle is substantially better (or worse). In those cases, it is the good one that is judged and quite possibly given a gold medal.
It’s as though the judges at Wimbledon decided that perhaps Federer and Nadal weren’t really playing at their best this year and deserved to be allowed another match to see whether they deserved to stay in the tournament. Or a restaurant critic giving Michelin stars to a restaurant where she has had to return a badly cooked dish.
At this stage of my fairly lengthy career, there is one thing of which I am sure. Every “Parker point” or medal, or media review represents the result of a single event: a single encounter or, possibly, in the case of a competition, a rapid succession of encounters with one or a small number of humans. It is not and cannot be definitive. Was Sabina Lisicki a 100-pointer in her losing-in-straight-sets Wimbledon final? Maybe not. But what if the clock had stopped
on the day she’d beaten Serena Williams. At the final, it was Marion Bartoli who got the magic score. Are either of these players really greater than either Williams sister?
So yes, critics are bullshitting if they pretend that the mark or medal they give a wine on any given day has the quality of a verdict from the Supreme Court. And the wine world is guilty of pseudo science when it talks of a 95-point or trophy-winner as if that non-scientifically awarded mark or prize defines it in a permanent fashion. The truly great wines – like the truly great sportsmen, actors, singers, artists and authors – are the ones that consistently impress a number of credible critics.