Don't let the LLM
pick a number.

Seven principles.

Each one rules out a way the LLM-pick-a-number failure mode sneaks back in. Together they define a scoring pipeline you can defend against an adversary who's read the prompt.

1. 01

   Separate observation from scoring

   The LLM finds evidence. A formula, not the LLM, produces the score. The number is computed, not chosen.

2. 02

   Discrete signed impact items

   Every piece of evidence gets one of {+5, +3, +2, +1, −1, −2, −3, −5}. Forces commitment. Removes the 7-out-of-10 anchor.

3. 03

   Diminishing returns (sqrt)

   normalized = net_impact / sqrt(total_items). The 40th item adds less than the 4th. Evidence farming is punished.

4. 04

   Density-weighted confidence

   Confidence = how much evidence the scorer found, not how sure the scorer feels. Sparse runs are visibly low-confidence.

5. 05

   Anchored center

   Sparse-evidence runs regress toward 50. The multiplier never exceeds 1.0 — high evidence confirms, never amplifies beyond raw.

6. 06

   Bounded scale with self-check

   Final scores live in [0, 100]. Across criteria, the spread must be ≥ 20 — otherwise the evaluator was not discriminating.

7. 07

   Separation of LLM and deterministic computation

   Independent passes by different model families collect evidence. Math, not the LLM, combines them. Adversarial synthesis catches contradictions.

Discrete impact → diminishing returns → confidence-weighted.

Two application patterns. Use the pooled variant for simple checkers (one bucket of evidence, one final score). Use the per-criterion variant for matrix benchmarks (formula runs once per criterion; weighted average produces the overall).

# Per criterion
net_impact         = sum(item.impact for item in items)
total_items        = len(items)
normalized         = net_impact / sqrt(total_items)
raw                = clamp(50 + normalized * 8.0, 0, 100)
density            = total_items / 20
multiplier         = 0.75 + 0.25 * clamp(density, 0, 1)   # never > 1.0
final              = round(50 + (raw - 50) * multiplier)
confidence         = clamp(density, 0, 1)

# Across criteria (overall)
overall_score       = round(sum(c.final * c.weight))
overall_confidence  = min(c.confidence for c in criteria)
self_check_span     = max(c.final) - min(c.final)
                      # must be >= 20

Discrete impact set: {+5, +3, +2, +1, −1, −2, −3, −5}. Hard cap of 5 items per perspective per criterion per pass. The multiplier never exceeds 1.0 (confirms, never amplifies). Sparse evidence is visibly low-confidence, not silently confident.

Run the math by hand.

Four small examples + one real-world rescoring. The toy cases show how the pieces interact; the rescoring shows what changes when an existing benchmark drops the LLM-picks-a-number step.

1. A

   Strong, abundant evidence

   given · net_impact = +25, total_items = 25

   • · normalized = 25 / sqrt(25) = 5.0
   • · raw = 50 + (5.0 × 8.0) = 90
   • · density = 25/20 = 1.25 → multiplier = 1.0
   • · final = 90, confidence = 1.0
2. B

   Strong, sparse evidence

   given · net_impact = +25, total_items = 4

   • · normalized = 25 / sqrt(4) = 12.5
   • · raw = 50 + (12.5 × 8.0) = 150 → clamped 100
   • · density = 4/20 = 0.2 → multiplier = 0.80
   • · final = round(50 + 50 × 0.80) = 90, confidence = 0.2
3. C

   Average

   given · net_impact = 0, total_items = 20

   • · normalized = 0
   • · raw = 50
   • · density = 1.0 → multiplier = 1.0
   • · final = 50, confidence = 1.0
4. D

   Weak, well-evidenced

   given · net_impact = −15, total_items = 25

   • · normalized = −15 / sqrt(25) = −3.0
   • · raw = 50 + (−3.0 × 8.0) = 26
   • · density = 1.25 → multiplier = 1.0
   • · final = 26, confidence = 1.0 (high confidence in a low score)

Three ways to use it.

One repo, three skills. Pick the one that matches your need. All three are skills.sh-installable into Claude Code, Cursor, Goose, OpenCode, and any other skills-aware agent.

// not sure? what-works-feedback-judge is the simplest. hackathon-judge if you have a code submission. evidence-scoring if you're bringing your own domain.