Add Human Approval to a High-Risk Action

What you will build

A small state machine that sits between an agent and a high-risk action. The agent can PROPOSE the action. It gets HELD. A human APPROVES or REJECTS it. Only an approved proposal is allowed to ACT. Every one of those steps writes a line to an audit record. The rule the whole lab enforces is simple. The agent can never take the high-risk action without a recorded approval.

Pick one real action to guard. Sending an invoice, issuing a refund, or sending an external email are all good choices. The pattern is the same for all three.

Why this lab matters

Some actions are cheap to undo and some are not. An agent that drafts a document is safe to run wide open. An agent that emails a customer, moves money, or changes a live record is not. When the agent gets those wrong, the damage is already out the door before you notice.

Human approval is the guardrail that separates the two. It is not about slowing everything down. It is about drawing one clear line: the agent proposes, a person decides, and the record proves who decided. That record is what turns “the agent did something weird” into “here is exactly what was proposed, who approved it, and what happened.”

Expected output

• A guarded action with four states: proposed, held, approved or rejected, acted.
• A gate that refuses to run the action unless the current proposal is approved.
• An audit record with one entry per state change, including who approved and when.
• A test run where a rejected proposal never fires the action.

Prerequisites

• You have read Chapter 7, Guardrails and Human Approval.
• You can run a small Python script and set an environment variable.
• You have one high-risk action in mind, real or stubbed.

Steps

1. Name the action and its inputs. For an invoice: recipient, amount, currency. Write these down. The proposal is these fields plus who is proposing.
2. Define the four states as an explicit set: proposed, approved, rejected, acted. No other path to acted exists except through approved.
3. Have the agent build a proposal and stop. It does not call the action. It hands back a proposal object and the state becomes held.
4. Write every transition to an audit log before anything else happens. Propose, approve, reject and act each append one line with a timestamp and an actor.
5. Put the gate in front of the action. The function that sends the invoice checks the state first. If it is not approved, it raises and writes a blocked entry to the audit log.
6. Run the human step by hand for now. Read the proposal, then call approve or reject. Later this can be a Slack button or an inbox reply, but the gate does not change.

Here is the shape of the state machine and its audit log. Keep it this small.

import json, time, uuid

def audit(record, event, actor, detail=""):
    record["log"].append({
        "at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "event": event, "actor": actor, "detail": detail,
    })

def propose(action, payload, actor="agent"):
    record = {"id": str(uuid.uuid4()), "action": action,
              "payload": payload, "state": "proposed", "log": []}
    audit(record, "proposed", actor, json.dumps(payload))
    return record  # HELD here. Nothing has happened yet.

def decide(record, approved, actor):
    if record["state"] != "proposed":
        raise ValueError(f"cannot decide from state {record['state']}")
    record["state"] = "approved" if approved else "rejected"
    audit(record, record["state"], actor)
    return record

def act(record, do_it):
    # The gate. The action cannot run any other way.
    if record["state"] != "approved":
        audit(record, "blocked", "system", f"state was {record['state']}")
        raise PermissionError("no recorded approval, refusing to act")
    result = do_it(record["payload"])   # the real high-risk call
    record["state"] = "acted"
    audit(record, "acted", "system", str(result))
    return result

Wiring it together, a rejected proposal never reaches the action:

def send_invoice(payload):
    # the real call would go here (email, Xero, Stripe, etc.)
    return f"invoice sent to {payload['to']} for {payload['amount']}"

p = propose("send_invoice", {"to": "acme@example.com", "amount": 4200})
decide(p, approved=False, actor="sam")   # human says no
act(p, send_invoice)                     # raises PermissionError, writes "blocked"

The model can draft the proposal payload, but it hands the object to propose and stops. It has no path to act. That separation is the guardrail.

Verification checklist

• A rejected proposal raises when you try to act, and never calls the real action.
• An approved proposal acts once, and a second act call is refused because the state is now acted.
• The audit log has an entry for propose, for the human decision, and for the act or the block.
• Every entry names an actor. The human decision is not attributed to the agent.
• You can read the full record and see what was proposed, who approved it, and what happened.

Common failure modes

• Letting the agent call the action directly “just this once” for testing. Once that path exists, it gets used.
• Approving the proposal but not checking the payload did not change between approval and action. Freeze the payload at propose time.
• Logging only the successful acts. The blocked and rejected entries are the ones that prove the guardrail works.
• Treating “the agent asked” as approval. Approval is a separate actor writing a separate entry.

Extension ideas

• Add a timeout. A proposal that is not approved within an hour expires and cannot be acted on.
• Move the human step to a Slack message with approve and reject buttons, keeping the same gate.
• Add a rule that auto-approves below a threshold, say invoices under 50 dollars, and logs the auto-approval as its own actor so it is still on the record.

Related chapter

Chapter 7, Guardrails and Human Approval.