Wine Scores: Noise Explained ( Part 2)

Seven Things We Could Actually Do Differently

A companion piece to The Noise in the Glass

Critique without remedy is just complaint — and I am as aware of that as anyone. The problems described above are real, but most of them are tractable. None of what follows requires formal training, specialist equipment, or a change of venue. Just a little more intention before and during the evening.

1. Define the construct before you taste

Before a single glass is poured, the convener should put a question on the table: are we ranking by quality, by typicity, by value, or by personal preference? These are different questions with different answers. Leaving the question implicit does not make it go away — it just means every member answers a different one in private. Five minutes of framing at the start of the evening is worth more than any methodological refinement afterwards.

2. Separate preference from quality

Ask members to produce two rankings: one for personal preference (which wine did you most enjoy?) and one for perceived quality (which wine do you think is the best example of what it is trying to be?). The gap between the two lists is often where the most interesting conversation lives. A wine can be technically excellent and not to your taste. A wine can be immediately lovable and not especially serious. Making that distinction explicit is one of the most illuminating things a tasting can do.

3. Run a simple calibration round

Before the main flight, serve two wines that are not part of the competition — one that is clearly straightforward, one that is clearly serious. Ask members to rank just those two. This is not a test. It is an anchor: a shared reference point that gives members a common sense of where the scale begins and ends. It takes ten minutes and costs almost nothing.

4. Record descriptors alongside rankings

Ask each member to write down two or three words for each wine before assigning a rank. Not a tasting note — just a word or two: “tobacco,” “too acidic,” “reminds me of a Côtes du Rhône.” This slows the process slightly and forces members to articulate what they are actually responding to. It also makes the post-scoring discussion far richer: instead of defending a number, members can compare the words.

5. Treat the discussion as data

The qualitative discussion after scoring is currently informal. It could be slightly more structured without losing its warmth. One approach: before the reveal, ask each member to name the wine they found most surprising and the wine they found most disappointing, and briefly say why. This draws out quieter voices without putting anyone on the spot, and it produces a richer record of the evening than a ranked list alone.

6. Name the outliers

When a wine receives a very high rank from one member and a very low rank from another, that disagreement is not a problem to be averaged away — it is the most interesting data point of the evening. The convener should flag those outliers explicitly: “This wine finished fifth overall, but two people ranked it first and one person ranked it last. What is going on there?” That question almost always produces the best conversation of the night.

7. Rotate the convener role

The convener shapes everything: the topic, the sequence, the framing, the discussion. There is no rule that says one person must do it all. Within a single evening, one member can introduce the topic, another lead the post-scoring discussion, a third handle the reveal. No single voice dominates the arc of the evening, and members who have prepared a piece of it listen differently to the rest.

None of these suggestions will eliminate the noise—and I would not want them to. The point is simply to know what we are doing and why. The scores will still be imperfect. The conversation will still go sideways. Someone will still rank the best wine last. But if we understand the evening a little better, we spend it a little better. That feels like enough.