A leaderboard tells you how a model performed on a fixed public test set. It does not tell you how that model will perform on your documents, your edge cases, or your definition of a correct answer. Those are different questions, and the gap between them is where most model-selection decisions go wrong.
Benchmarking against your own data replaces a ranking with a measurement. The measurement is frequently uncomfortable, usually cheaper than expected, and almost never matches the leaderboard order.
What a leaderboard actually measures
Public benchmarks measure general capability on a curated task, scored in aggregate, on data the model provider did not choose but that thousands of people have now optimised against. They're useful for what they were built for: comparing broad capability across model generations.
For this workflow, on this input distribution, at this volume, which configuration produces acceptable output at the lowest cost? Note that the question says "configuration," not "model." That's deliberate, and we'll come back to it.
Four reasons leaderboard rank doesn't transfer
Distribution mismatch
Benchmark inputs are clean. Yours are not. A model scored on well-formed documents tells you little about a receipt shot at an angle, in mixed languages, with a thumb over the corner.
Aggregate scores hide subgroup failure
A model at 91% overall can be at 99% on your common case and 40% on the document type that carries the most financial exposure. The average looks fine; the system is not.
Contamination
Widely published test sets end up in training data, so a high score may reflect exposure rather than capability. Your own data has been published nowhere; that's the point of it.
Task is not workflow
You deploy a model inside a prompt, a preprocessing step, a validation layer, at a temperature, with a schema. The model is one variable among several, and often not the dominant one.
On our tourist tax refund platform, that real-world input distribution is the problem: the system runs at 94% OCR accuracy and 92% classification accuracy on receipts photographed at an angle, in mixed languages, with a thumb over one corner. No public benchmark predicts that number.
What we built to do this properly
We built our automated benchmarking tool because doing this by hand doesn't scale past the first afternoon. It tests models, prompts, and architectures as combinations rather than one at a time.
The throughput number matters more than it first appears. At 10 cases per run, you can only afford to test the two or three options you already believe in, which means your evaluation confirms your prior instead of challenging it. At 1,000+, you can test the configuration nobody in the room expected to win, and that's usually the one that does.
How to build an evaluation set worth trusting
- 1Sample from production, not the happy path.Pull real inputs, including ones that previously failed. A set of only working documents is a regression test, not a benchmark.
- 2Include the known-hard cases deliberately.Every operations team has a mental list of inputs that break things. That list is your highest-value test data.
- 3Define correctness per field, not per document."Got this invoice right" isn't measurable. "Vendor name exact match, total within zero tolerance, date normalised to ISO" is.
- 4Label it, and label it once, carefully.The labelled set is the asset. Models change monthly; a well-labelled evaluation set stays valid for years.
- 5Weight by consequence, not by frequency.If one document type is 3% of volume and 60% of financial exposure, it does not get 3% of your test set.
- 6Hold a portion back.If every case is visible during prompt iteration, you'll tune to the test set and learn nothing about generalisation.
Step 4 is where most of the effort goes and where most projects underinvest. It is also the only step that can't be automated away, because it encodes what your business considers correct.
What you tend to find
Three patterns recur often enough to expect them.
The cost consequence
This is the part that reaches the budget. Model choice sets your per-transaction cost, and per-transaction costs compound with volume in a way that a monthly pilot invoice conceals. A workflow processing a few hundred documents in testing and a few hundred thousand in production has a different economic profile entirely, and the decision that determined it was made during evaluation.
Choosing on measured fitness rather than leaderboard position is the first move in most of our AI cost optimization work, and it's the one that changes the run rate rather than trimming around the edges. The second is architectural: keeping model calls per transaction as low as the accuracy requirement allows, which is a question for AI architecture rather than model selection.
Not the model you picked this quarter, but the ability to re-decide cheaply every quarter after.
It's a regression net, not a one-time exercise
The mistake after a successful evaluation is treating it as finished. Models get deprecated. Providers ship silent updates. Your input distribution shifts when a client changes their document template or you enter a new market.
A maintained evaluation set turns each of those from a discovery, usually made by a customer, into a scheduled test run. That's the durable return on the labelling effort.