Skip to main content

10 · Agents and AI engineering

Tool calling and harnesses

The model asks, your program executes: how a model uses tools, and the layer of software that decides what actually runs.

13 minCase study: docket (opens in new window)

A language model only produces text. For a system to do something, like read a file, query a database or open a pull request, that text has to be connected to code that runs. The standard way is , and its founding rule fits in one line: the model asks, your program executes.

Almost everything that matters in an is decided at that boundary. This lesson covers four ideas: what a tool is, what must happen between a request and its execution, what a is and why it matters who owns the loop, and why every action should pass through one checkpoint.

What a tool is

Think of a restaurant menu: diners read names and descriptions and order, but never enter the kitchen. A tool has four parts, and only the first three, the menu, reach the model:

Part What it is Who uses it
Name a unique label, such as read_file the model, to ask for it
Description when and why to use it the model, to decide
Input schema a formal description of the accepted inputs the model writes inputs, the system checks them
Handler the code that runs only the system

The handler is the kitchen: it never leaves your program. Everything else in this lesson rests on that split.

From request to execution

To use a tool, the model replies with a request: the tool's name and its inputs, called arguments. Before anything happens in the world, a careful system goes through five steps:

  1. Read the arguments. They arrive as text that should hold JSON (a standard data format), and sometimes doesn't.
  2. Validate. Does the tool exist? Are the required inputs there?
  3. Authorize. May this agent run this call with these inputs?
  4. Execute the handler, with a time limit.
  5. Report back the output as a result tied to the original request, for the model's next round.

Demos tend to skip steps 1 to 3. Serious systems put most of their logic there.

When in doubt, refuse

The model can write broken arguments: invalid JSON, a list instead of an object, or nothing at all. The rule is to fail closed: if you cannot read a request, you cannot judge it, and an unjudged request must not run.

Errors are results too

The tool ran and failed is useful information the model can act on. The tool did not run because it was not allowed is a safeguard working. A naive system mixes them up; a good one reports both to the model in words, because silence makes the model retry blindly. The result must also carry the request's identifier, or many providers reject the whole conversation. docket's results state which case happened (refused and why, awaiting approval, failed, or the output) and are built from the original request, so they can't be attached to the wrong one.

Several calls at once

One reply can ask for several tools. That does not mean they must run at the same time: independent reads can, two writes to the same thing should not. docket runs them one after another, in order, checks its tool-call limit against the whole batch first (see From LLM app to agent), and saves the request and all its results as one unit.

What a harness is

All of the above takes a lot of code unrelated to the task: limits, saving state, permissions, approvals, logging, context, provider quirks. That code is the harness.

Read the layers from top to bottom, from pure text to real consequences:

  1. Modelpredicts text; runs nothing
  2. Agentmodel + goal + decision loop
  3. Harnessruns the loop, decides what is allowed
  4. Tools and environmentfiles, processes, APIs, money
Each layer down has more real consequences. The harness is the last point where a request can still be refused.

Products like Claude Code are agent and harness at once. Separating the two lets you ask the key question: who has the last word when the model asks to run something?

Owning the loop or wrapping it

An agent framework usually runs the loop for you, so your controls can only live where it lets you plug in (, callbacks). If there is no place to plug in before each action runs, you cannot decide there.

  • Wrapping a loopthe framework runs each turn
    • Your checks go where hooks exist
    • Maybe none before the action
  • Owning the loopyour code runs each turn
    • Checks before every action
    • Standard protocols reused
Whoever runs the loop decides where checks can go. Owning it is the only way to be sure one runs before every action.

Tested, documented policies that protected nothing: that is the risk of wrapping someone else's loop without checking where you can step in. Using a framework is not wrong in itself.

One checkpoint for every action

If there are two ways to run a tool and only one has controls, the system has no controls. Every tool call should go through a single , one function that holds all the checks.

In docket

  1. Cancelled?if so, stop before anything runs
  2. Tool and argumentsunknown or unreadable means refused
  3. Policy decisionallow, ask or deny; strictest wins
  4. Human approvalonly when the answer is ask
  5. Run the handlera crash becomes a result
The order inside docket's single checkpoint. Any refusal goes back to the model as a result, in words, and never reaches the handler. Source: Security: Layered & Convention-Based · core/tools.py

A comment saying "don't add another path" can be ignored, so docket turns it into a test. The test reads the project's source code and fails if any module outside a short allowed list imports the handlers. A guard like this covers exactly what it encodes, one kind of shortcut, not every conceivable one. More in Testing and evals for agents; allow/ask/deny decisions are in Governance and human-in-the-loop.

Putting it into practice

  • Build or adopt the harness deliberately. Building gives full control of every checkpoint but means maintaining code that doesn't set you apart; adopting is faster, but your governance ends where the vendor's hooks end.
  • Ask for proof, not a feature list: see a deny rule actually block a call in your own environment.
  • Make risky tools safe to repeat, because the model may ask twice.
  • Keep one path to execution, and test that it stays one.

Go deeper

docket's documentation records how it built and verified this boundary:

Key takeaways

  • A tool is an advertisement (name, description, input schema) plus a handler that never reaches the model.
  • Reading, validation and authorization sit between request and execution; when in doubt, refuse.
  • Refusals and errors go back to the model in words, tied to the request they answer.
  • Whoever owns the loop owns the points where you can intervene; wrapping someone else's loop can leave your policies doing nothing.
  • A single chokepoint is only real if something verifies it, such as a test that reads the code.

Check yourself

A tool request arrives with arguments that are not valid JSON. Why refuse it rather than guess?

If the system can't read the request, it can't judge whether it is allowed, and an unjudged request must not run. Keeping the raw text also lets the audit trail show exactly what the model asked for.

A framework offers policy hooks, but they only run after each tool call. What is the problem?

A policy that runs after an action can observe it but never stop it. docket hit a version of this: while it wrapped a loop it did not own, its policies were never evaluated, which is why it chose to own the loop.

A test checks that only a few allowed modules import the tool handlers. What does it guarantee, and what not?

It guarantees the handler module is only reached through allowed places, such as the chokepoint. It cannot catch paths that avoid that import: docket's web-fetch handler lives in its own module, and importing it directly elsewhere would not fail the test.