Build Guardrails and Human Approval

The real business problem

An agent that can only read is safe and mostly useless. The value shows up the moment it can act, which is the moment it can also do harm. It can email the wrong client, refund the wrong customer, delete a record you needed, or send an invoice with a number that is off by a zero.

The instinct is to solve this with trust. You watch it for a week, it behaves, so you let it run. That is not a control. That is luck with a delay.

The real control is deciding, in advance, what the agent is allowed to do on its own, what it may do as long as it leaves a trail, and what it must never do without a human saying yes first. This chapter shows you how to draw those lines and how to build the approval step that enforces the top one. After this you will be able to point at any action your agent takes and say who signed off on it and why.

The core idea

Not every action carries the same risk, so not every action should get the same freedom. Sort actions onto a ladder with three rungs.

• Low risk. Reversible and internal. Reading data, drafting a summary, categorising a ticket, proposing a reply. If it gets this wrong, you delete the output and move on. The agent just does it.
• Medium risk. Reversible but it touches something real. Updating a CRM field, moving a file, posting to an internal channel, creating a draft invoice. The agent does it, but it writes a log entry so you can see what changed and undo it.
• High risk. Irreversible, outward facing, or expensive. Sending an external email, paying or refunding money, deleting records, changing a price, signing anything. The agent may prepare the action, but a human approves it before it happens.

The whole skill is putting each action on the right rung. Too low and you get harm. Too high and you get an agent that needs a human for everything, which is just a slow human.

A simple model

RISK LADDER                 APPROVAL LOOP (high risk only)
low     -> ACT              1. PROPOSE  agent writes the exact action
medium  -> ACT + LOG           |
high    -> HOLD ----------> 2. HOLD     action is frozen, nothing sent
|
3a. APPROVE ---> 4. ACT   run it, log the result
3b. REJECT  ---> stop, record the reason
(or timeout expires -> stop)

The approval step is a small state machine, and treating it as one is what keeps it honest. The agent does not send and then ask forgiveness. It writes down the exact action it wants to take, freezes it, and waits. A human approves or rejects. Only an approval moves it to the act state. Silence is not approval. If nobody answers by the deadline, the proposal expires and nothing happens.

A concrete example

An agent handles accounts receivable. It drafts the monthly invoice for each client and is meant to email them out.

Drafting the invoice from the timesheet data is low risk, so it just does that. Saving the draft and updating the account record is medium risk, so it does that and logs it. Emailing the invoice to the client is high risk, because it is outward facing and it is about money, so it holds.

What it holds looks like this. The recipient is priya@northwind.example. The subject is “Northwind invoice, July.” The attachment is invoice-1042.pdf, total 4,200 dollars, which is an illustrative figure. The agent presents that exact package and waits. A person reads it, notices the total should have been 2,400 dollars because half the month was on a fixed fee, rejects it with that reason, and the agent revises. Nothing left the building. The audit trail now shows the catch and the correction, which is worth as much as the fix.

The implementation pattern

Build the approval step as a record, not a pause in a script. A pause in a script dies when the process restarts. A record survives, and you can see it, query it, and audit it later.

When the agent hits a high-risk action, it does not call the tool. It writes a proposal and stops. The proposal captures everything a human needs to decide without going digging.

{
  "id": "prop_8842",
  "action": "send_email",
  "risk": "high",
  "payload": {
    "to": "priya@northwind.example",
    "subject": "Northwind invoice, July",
    "attachment": "invoice-1042.pdf",
    "amount": 4200
  },
  "reason": "Monthly invoice run, client on time-and-materials",
  "status": "hold",
  "expires_at": "2026-07-25T17:00:00Z",
  "created_by": "ar-agent",
  "created_at": "2026-07-23T09:12:00Z"
}

The rules that make this safe are boring on purpose.

• The action list is a whitelist, not a blacklist. The agent can only call tools you have granted it. A high-risk tool checks for an approved proposal id before it runs and refuses without one. That refusal lives in the tool, not in the prompt, because a prompt can be argued with and a code check cannot.
• Approve or reject is a separate step by a separate identity. The thing that approves is not the agent. A person clicks approve, or a second system with its own authority does. The approver’s identity gets stamped on the record.
• Rejection carries a reason, and the reason goes back to the agent so the next attempt is better, not a blind retry of the same mistake.
• Proposals expire. A held action that nobody answers should die, not linger for a week and fire when someone finally clicks. Set a deadline and let silence mean no.

Now handle the case where the agent is not confident enough to even propose. This is escalation, and it is the other half of guardrails. Give the agent explicit conditions to stop and ask rather than guess.

• It is missing information it needs, like a client with no email on file.
• It hits conflicting instructions, like two rules that both apply and disagree.
• Its own confidence in the next step is low, and you have told it that low confidence on a real action means hand it to a person.
• The action it wants is outside its allowed set entirely.

In every one of those, the recovery path is the same shape. Stop, describe the situation plainly, hand it to a named human or queue, and do not proceed until you get an answer. An agent that guesses when it is stuck is more dangerous than one that does nothing, because the guess looks like work.

The last piece is the audit trail, and it is not optional the day something goes wrong. Every action, proposed or taken, writes one line. What was done, by which agent, when, on what inputs, and on whose approval. When a client calls and asks why they got a strange invoice, you should be able to answer in one query, not one afternoon. The trail is also how you tune the ladder. If a medium-risk action keeps causing cleanups, promote it to high. If a high-risk action gets approved unread every single time, ask whether it was ever really high risk, or whether your people have stopped reading.

Decision rules

Related lab

Add Human Approval to a High-Risk Action. You will take one real action from your own workflow, most likely an outbound email or a payment, and wrap it in the propose, hold, approve or reject, then act loop. You will see the agent stop at the gate, present the exact action, and refuse to proceed until you sign off. It is the smallest complete version of everything in this chapter, and it is the pattern you will reuse for every high-risk tool after.

Practical checklist

What changes after this chapter

You stop trusting the agent because it has behaved and start trusting the system because it cannot misbehave in the ways that matter. The freedom the agent has is now a decision you made on purpose, written down, and enforced in code, not a habit that grew because nothing bad happened yet. That is the difference between an agent you can defend to a board and one you are quietly hoping stays lucky.

Next: Chapter 8, Verify the Work Was Actually Done.