Governance and human-in-the-loop

An agentAgent A system in which a model, in a loop, chooses the next action until it decides it is done or hits a stop condition. that only chats needs little oversight. One that runs commands, edits files or deploys software does: something has to decide which of its actions happen, under which rules, and record what was decided. The usual instinct is to write the rules in a file like AGENTS.md and trust the model to follow them.

This lesson explains why that is not enough, through five ideas: the difference between an instruction and a policy, how to combine several rules safely, why removing a tool beats forbidding it, what a good human approval looks like, and what should happen when nobody answers.

Instruction versus policy

You can tell a new employee "never wire money without a second signature", or set up the bank account so no wire leaves without one. The first depends on memory and judgment; the second holds even if someone tricks them. Agents face the same choice.

An instruction ("don't push to the main branch") is text the model reads. It works only if the model remembers it, understands it and ranks it above other text, including text an attacker slipped in. A policy is code in the harnessHarness The software around the model that runs its loop, carries out the tools it requests, prepares what it sees and enforces limits., the program that runs the agent, that checks every action before it happens, whatever the model wants.

When a policy blocks something it leaves a record, and it can be tested like any other code. Instructions improve the normal case; policy bounds the worst case. You need both. The core idea: the model must not be the security boundary.

Deny, ask, allow: the strictest rule wins

A policy gives one answer per action: deny, ask a person, or allow. When several rules match, the robust way to combine them is simple: the most restrictive answer wins. That way, adding a rule can never reopen something another rule closed.

Remove the tool instead of watching it

Asking a person about everything does not work, so first sort actions by risk: reads (the risk is leaking data), local changes (usually reversible) and high-risk actions with external or irreversible effects. With a generic tool like a shell, the risk is in what is run; Tool calling and harnesses covers how commands are classified.

Then tell apart two kinds of control:

• A permission means the agent has the tool and a check inspects each use. If the check has a gap, the action happens.
• A removed capability means the agent does not have the tool at all. There is nothing to check: a wall, not a guarded door.

The working rule: a role that cannot do something is safer than one that was told not to. Where you cannot remove the tool, check the permission at one single point.

What a good approval looks like

Some decisions no rule can make alone: is this production push the planned deploy or a mistake? For those you ask a person, and a serious design answers five questions: what triggers the request, what the person sees, which channel carries it, what happens on a "no" or on silence, and who decided.

The channel is an attack surface too. docket's Telegram bot accepts only four commands, from chats linked in advance, and records anything else as refused. Its HTTP API requires a tokenToken A small chunk of text, often part of a word, that a model reads and writes and that providers charge by., but the caller states which channel it is, so the channel in the audit logAudit log A record of who decided what, when and under which rule; linking each entry to the previous one with a cryptographic fingerprint (hash) makes later edits detectable. is a claim, not proof.

When nobody answers

An approval granted automatically after a timeout is not a control, just a delay: an attacker only has to act when nobody is watching. The rule: a control that fails must fall back to the safe behavior, which here means deny.

Keeping these cases apart pays off: "nobody was asked", "someone said no" and "nobody answered in time" tell you about configuration, the action and the team's availability, respectively.

Putting it into practice

• Decide the level of oversight per action. Some need approval first, some run under supervision with a way to stop them, some are reviewed afterwards. Grant more autonomy only as evidence builds up: traces, evaluations, approval history.
• Keep rules in versioned, tested files, not in promptsPrompt Everything sent to the model in one call: instructions, conversation messages and any tool definitions..
• Watch the approval numbers. If nearly everything is approved within seconds, the approvals are decorative.
• Decide what happens at 3 a.m. when nobody is available. The safe answer is that nothing risky runs.

Go deeper

docket's documentation describes each of these mechanisms in more detail:

• Security: Layered & Convention-Based lists docket's security layers, from the instructions agents read to the tool-call check that cannot be switched off, and says what each one does not cover.
• Agent Teams (Pods) explains how each role is declared as data, including its edit rights and the tools it may use.
• DOCKET Architecture shows where the policy engine runs: when a task is queued, before each tool call and on the output of each step.

Key takeaways

• An instruction improves the normal case; a policy enforced by code bounds the worst case. The model is not the security boundary.
• When rules disagree, the strictest wins. Check that every failure path really fails closed.
• Removing a tool from a role is stronger than forbidding its use.
• A useful approval is atomic, hides secrets, records who decided and how, and denies when nobody answers.
• Ask people only where their judgment adds something; otherwise approvals become a rubber stamp.

Check yourself