Testing and evals for agents

Ordinary code is easy to check: if a function given 2 and 2 does not return 4, there is a bug. An agentAgent A system in which a model, in a loop, chooses the next action until it decides it is done or hits a stop condition. does not work that way. Between the request and the result sit a model's decisions, which cannot be read like code, rarely have one right answer, and shift whenever the promptPrompt Everything sent to the model in one call: instructions, conversation messages and any tool definitions., the tools or the model change.

The answer is to split the system in two. Almost everything around the model (the loop that calls it, the rules on what it may do, the code that reads its replies) behaves the same way every time, and is tested like any other software. The model's decisions are evaluated, with different tools and expectations.

This lesson covers four ideas: why tests and evalsEval A measurement of how well the whole system behaves on realistic cases, including the steps it took along the way, scored with imperfect checks and read with statistics. differ, how to stand in for the model during tests, how to protect the design itself, and what an agent eval should measure, including why a check that cannot fail is worse than none.

Tests and evals are different things

The two common mistakes mirror each other: testing everything against a real model with exact checks (slow, expensive tests that break at random), or never running against a real model at all (green tests, broken product). Tests cannot prove the agent solves tasks; evals will not reveal a code path that skips the permission check.

Standing in for the model

To test the code around the model, replace the model with a stand-in that follows a script: it returns a fixed list of replies, including requests to use tools, and records what it was sent. That is a fake modelFake model A scripted stand-in for a real model, so tests of the code around it give the same result every time.. It lets you trigger on purpose what a real model does only rarely, such as broken output, a tool that does not exist or an attemptAttempt One numbered try at running a piece of work, with its own expiring lock, a counter that blocks outdated writes and its own working folder. to write outside its folder, and check that the system reacts correctly.

That is a good general rule: fake the model, not the protocol. If you also fake the network layer and the message format, you will not notice the day the provider changes a field.

Protecting the design, not just the behavior

Some guarantees are about how the system is built. A common one: "every tool call goes through a single checkpoint", where permissions are checked and the action recorded. A normal test proves today's path goes through that checkpoint. It cannot prove nobody added a second path that skips it.

For that, a test can read the source code itself and fail if the rule is broken, for example if the checkpoint is called from more than one place. These tests guard architectural invariants: rules about the shape of the code that must stay true as it grows.

The places where two parts meet need their own checks. When another program reads your output, its format is a promise, so docket fails a test if its published output schema drifts from the code by a single byte (Harness mode (ADR 0001)). And two parts tested separately can still fail together: in docket, one change wrote a refusal as plain text while another read it back with a pattern, and together they lost the details. The fix was one function that owns the format, plus a test that runs the real writer and the real reader together.

What an agent eval measures

An agent eval looks at several things at once:

• Outcome: is the final result correct? For code: it builds, the project's tests pass, and so do hidden tests the agent never saw.
• Trajectory: was the path reasonable? Did it touch forbidden files, try dangerous commands, or edit a test so it would pass?
• Cost: turns, tokensToken A small chunk of text, often part of a word, that a model reads and writes and that providers charge by. and time.
• Safety: how it handles hostile cases, such as instructions hidden in the data it reads.

Two agents given the same task. Look at the path each one took, not just the result:

Agent A: search → read → edit → run tests → done             5 steps  12k tokens
Agent B: read ×14 → rewrite the failing test so it passes
         → edit code → run tests → done                     21 steps  80k tokens

An outcome-only eval scores both the same. A trajectory eval finds that B cost six times more and "passed" by changing the test meant to judge it.

When no exact check exists ("is this answer helpful?"), a common approach is LLM-as-a-judgeLLM-as-a-judge Using a model to grade outputs; trustworthy at scale only after its grades have been checked against human ones.: another model grades the result.

That removal is the lesson: an eval that cannot fail is worse than no eval, because it produces false confidence. For evaluating models themselves, see Evaluating models.

Putting it into practice

• Run both halves. Fast, exact tests on every change; real-model evals as often as the budgetBudget A hard limit on tokens, calls, time or money, enforced before the resource is spent. allows, such as a small set on every change and the full set before switching models.
• Look at the path, not only the score. A success rate hides agents that cheat or waste.
• Turn every production incident into an eval case, so the same failure cannot return unnoticed.
• Make sure every check can go red. Break what it guards once, on purpose, and watch it fail.

Go deeper

docket's documentation shows how it separates evidence about its own code from evidence about models:

• Adoption evidence publishes the results of the scripted scenarios, counts safe stops as failures, and says plainly what they do not measure.
• Harness mode (ADR 0001) explains why docket's output format is versioned for an outside program and pinned by a test that regenerates the schema and compares it.
• Development harness and context policy describes how changes to agent skills are graded: a fresh agent does the task, and a rubric it never sees judges the files, diffs and exit codes, not the prose.
• Models, gateways, and coding harnesses advises choosing a model only after evals on your real tools and tasks, and keeps live-model checks opt-in.

Key takeaways

• Test what behaves the same every time; evaluate the model's decisions with rates over many runs.
• Replace the model with a scripted fake in tests, but keep the real message format.
• Tests that read the code protect design rules, such as a single checkpoint, that behavior tests cannot.
• An agent eval checks outcome, trajectory, cost and safety; a model judge needs calibration first.
• An eval or guard that cannot fail produces false confidence.

Check yourself