Follow-up from the per-test-case diffs in the eval results view.
GET /eval/runs/{id}/pairs resolves the overall winner (PairVerdict.WinnerVariant) but leaves PairVerdict.Dimensions as the raw letters the judge saw — {"correctness": "b"} (internal/eval/pairview.go:29-44, values written verbatim by eval_verdict, internal/mcpserver/tools_eval.go:33-42).
The operator-facing results view therefore has a per-dimension map it cannot attribute to a model. EvalTaskDiffs.svelte currently works around this client-side: a non-tie verdict on an item is the key for that item’s letters (its winner letter is its winner_variant, so the other letter is the other side of the pair), and the raw letter is kept when every verdict on the item is a tie.
That workaround reconstructs a mapping the server already holds. PairDetails has the pair’s Assignment and each item’s presentation_order, so it can resolve dimensions the same way it resolves the winner — via VariantFor (internal/eval/pairing.go:253) per dimension value.
Proposed: add a resolved map alongside the raw one on PairVerdict (e.g. dimensions_variant, tie values preserved as tie), keeping the raw letters for anyone auditing what the judge actually saw. Then drop the client-side resolver.
This is read-side only and does not weaken blinding: the assignment stays server-side, and the pairs endpoint is already the unblinded operator view (never reachable from the judge’s MCP tools).
Follow-up from the per-test-case diffs in the eval results view.
GET /eval/runs/{id}/pairsresolves the overall winner (PairVerdict.WinnerVariant) but leavesPairVerdict.Dimensionsas the raw letters the judge saw —{"correctness": "b"}(internal/eval/pairview.go:29-44, values written verbatim byeval_verdict,internal/mcpserver/tools_eval.go:33-42).The operator-facing results view therefore has a per-dimension map it cannot attribute to a model.
EvalTaskDiffs.sveltecurrently works around this client-side: a non-tie verdict on an item is the key for that item’s letters (its winner letter is itswinner_variant, so the other letter is the other side of the pair), and the raw letter is kept when every verdict on the item is a tie.That workaround reconstructs a mapping the server already holds.
PairDetailshas the pair’sAssignmentand each item’spresentation_order, so it can resolve dimensions the same way it resolves the winner — viaVariantFor(internal/eval/pairing.go:253) per dimension value.Proposed: add a resolved map alongside the raw one on
PairVerdict(e.g.dimensions_variant, tie values preserved astie), keeping the raw letters for anyone auditing what the judge actually saw. Then drop the client-side resolver.This is read-side only and does not weaken blinding: the assignment stays server-side, and the pairs endpoint is already the unblinded operator view (never reachable from the judge’s MCP tools).