Evals are a repeatable set of test cases, with a way of scoring the output, that tell you whether a change to your prompt, model or retrieval made the system better or worse. They are to AI features what a test suite is to ordinary code.
Also called: evaluation, benchmarks · Reviewed
Start from failures you have actually seen. Every complaint, every wrong answer, every awkward edge case becomes a case in the set, paired with what the right output would have been. Twenty real cases are worth more than a thousand synthetic ones, because they represent the distribution your users actually produce. Twenty to fifty cases drawn from real failures is enough to be useful on day one; a suite of five hundred synthetic cases that nobody trusts is not.
Then choose a grader per case. Some can be checked exactly — did it return valid JSON, did it contain the right identifier, did it refuse when it should have. Others need a rubric, scored by a person or by another model prompted with clear criteria. Model-as-judge is useful and imperfect; spot-check its scores against your own.
Without evals, every change is a guess. Prompt edits interact in ways nobody predicts, and improving the case in front of you while quietly breaking three others is the normal outcome, not the unlucky one. A suite you can run in minutes is what turns iteration from opinion into evidence.
It is also the clearest line between a demo and a product. Demos are judged on their best output; products are judged on their worst.
Trusting public benchmarks as a proxy for your task. A model that leads a leaderboard may well be worse at your specific extraction job, and the only way to know is to run your own cases.
The other is leaving it too late. Teams that build the eval set after launch spend the intervening months unable to answer "is this better?" about any change they make.