You can't grade clinical reasoning with a model judge

The grader is close to a coin flip

Start with the headline finding. When independent work tested whether LLM judges can separate complete from incomplete clinical answers, they landed at an AUC of 0.49 to 0.66, from no better than chance to only marginally above it. And agreement on the label is not the same as agreement on the reasoning: even when a model judge and a clinician both call an answer incomplete, they cite the same missing element just 24.6% of the time. Three-quarters of the time the grader reaches the "right" verdict for a different reason than the clinician would.

That second number matters more than it looks. A judge that gets the label right by accident is pattern-matching to surface features that happen to correlate with completeness in the test set, not measuring clinical reasoning. Optimize a model against that signal and you train it to satisfy the judge's shortcuts, not to reason well. The grader's blind spots become the model's objective.

HealthBench shows how expensive the human bar is, and where the model falls short of it

HealthBench is the most serious attempt to grade clinical conversation at scale, and it is instructive because it is careful. Its rubric took 262 physicians across 60 countries writing 48,562 bespoke criteria to build, then it grades answers against those criteria with a model grader (HealthBench, OpenAI, 2025). The scale of the human effort is the point: this is what it costs to specify what a good clinical answer looks like.

But the grader that applies those criteria only agrees with physicians about as well as physicians agree with each other: a macro-F1 near 0.709, in an agreement band roughly 55 to 75 percent. Read charitably, that says the automated judge is roughly at the level of a single human rater. Read carefully, it says the judge inherits all the noise of a single rater with none of the recourse: when two physicians disagree you can adjudicate, but a model grader gives you one opaque verdict, and on HealthBench 94.1% of cases carry only two graders' worth of human signal to check it against.

None of this is a knock on HealthBench, which is honest about its own limits. It is a demonstration that even the best-resourced model-grading pipeline in medicine tops out at single-rater reliability, and that the difficulty is real, not artificial. On HealthBench Hard, no frontier model scored above 32% at release. The ceiling is low because the task is hard; a grader that barely beats chance cannot tell you reliably which model is closest to clearing it.

Grading is physician work, and the grades are the product

If the judge is unreliable, the leaderboard is unreliable, and no amount of prompt engineering fixes a grader that is near chance on the underlying task. The signal has to come from clinicians: physicians reading the answer, applying an adjudicated rubric, and grading correctness, completeness, safety, and calibration, with more than one physician per criterion so disagreements get resolved instead of averaged into noise.

There is a second, sharper point. The human calibration signal is the valuable thing, not overhead on the way to a model grader. Every automated judge in this space is ultimately calibrated against human ratings; that is the ground truth the whole enterprise is trying to approximate. Physician-graded conversations are that ground truth, delivered directly. Instead of training a model to imitate a cheaper judge that lands near a coin flip, you get the physician signal the judge was only ever a lossy proxy for.

A leaderboard is only as trustworthy as its grader. On clinical reasoning, the model grader is not trustworthy, and the way out is a physician, not a better prompt.

See our rubric-conversations track for physician rubric-graded clinical conversations.