Human in the loop

Some actions are too important to leave to a model: sending an email to a customer, issuing a refund, deleting data. Human in the loop means the agent proposes these actions and a person approves, edits, or rejects them before anything happens. This lesson shows how to classify tools by risk, enforce approval in code rather than in the prompt, record every decision, and pause an agent until a reviewer responds.

Last updated: 01 Nov, 2026

Why humans stay in the loop

Definition · Human in the loop (HITL)

A design where certain agent actions require explicit human approval before execution. The model can propose; only a person can authorise.

Models make mistakes — wrong recipient, wrong amount, misunderstood request.
Inputs can be malicious — a prompt injection in an email can ask the agent to forward private data.
Some actions cannot be undone — money sent and emails delivered stay sent.
Accountability — regulated decisions need a named person who approved them.

Classify tools by risk

Tier	Examples	Policy
Read-only	`search_docs`, `get_order_status`	Run automatically
Reversible write	`create_draft`, `add_label`	Run automatically, log, allow undo
Sensitive or irreversible	`send_email`, `issue_refund`, `delete_record`	Require human approval
Forbidden	`change_permissions`, `run_shell`	Never exposed to the model

Watch out

“Ask the user before sending emails” in a system prompt is a request, not a control. A model can ignore it or be tricked out of it. Approval must be enforced in the code that executes tools, where the model cannot bypass it.

An approval gate in code

Python

import json
from datetime import datetime, timezone

POLICY = {
    "get_order_status": "auto",
    "issue_refund": "approve",
    "send_email": "approve",
}

AUDIT_LOG: list[dict] = []


def get_order_status(order_id: str) -> dict:
    return {"order_id": order_id, "status": "delivered", "paid": 49.0}


def issue_refund(order_id: str, amount: float) -> dict:
    return {"refunded": amount, "order_id": order_id}


def send_email(to: str, body: str) -> dict:
    return {"sent_to": to}


IMPLEMENTATIONS = {"get_order_status": get_order_status, "issue_refund": issue_refund, "send_email": send_email}


def execute(name: str, args: dict, approver) -> dict:
    policy = POLICY.get(name, "forbidden")
    if policy == "forbidden":
        result = {"error": f"tool {name} is not allowed"}
    elif policy == "approve":
        decision = approver(name, args)          # {"action": "approve" | "edit" | "reject", ...}
        if decision["action"] == "reject":
            result = {"status": "rejected by reviewer", "reason": decision.get("reason", "")}
        else:
            args = decision.get("args", args)    # an edit replaces the arguments
            result = IMPLEMENTATIONS[name](**args)
    else:
        result = IMPLEMENTATIONS[name](**args)
    AUDIT_LOG.append({"tool": name, "args": args, "policy": policy, "result": result,
                      "at": datetime.now(timezone.utc).isoformat(timespec="seconds")})
    return result

The reviewer is just a function, so it can be a terminal prompt, a Slack button, or a web page. Here a scripted reviewer stands in for a person:

Python

def reviewer(name: str, args: dict) -> dict:
    print(f"  approval requested: {name}({json.dumps(args)})")
    if name == "issue_refund" and args["amount"] > 49.0:
        print("  reviewer: amount is more than the order total -> edit to 49.0")
        return {"action": "edit", "args": {**args, "amount": 49.0}}
    if name == "send_email" and not args["to"].endswith("@example.com"):
        print("  reviewer: unknown recipient -> reject")
        return {"action": "reject", "reason": "recipient is not the customer"}
    print("  reviewer: approve")
    return {"action": "approve"}


print(execute("get_order_status", {"order_id": "A-1001"}, reviewer))
print(execute("issue_refund", {"order_id": "A-1001", "amount": 490.0}, reviewer))
print(execute("send_email", {"to": "[email protected]", "body": "Here is the data"}, reviewer))
print(execute("delete_record", {"id": 7}, reviewer))
print(len(AUDIT_LOG), "audit entries;", [e["policy"] for e in AUDIT_LOG])

{'order_id': 'A-1001', 'status': 'delivered', 'paid': 49.0} approval requested: issue_refund({"order_id": "A-1001", "amount": 490.0}) reviewer: amount is more than the order total -> edit to 49.0 {'refunded': 49.0, 'order_id': 'A-1001'} approval requested: send_email({"to": "[email protected]", "body": "Here is the data"}) reviewer: unknown recipient -> reject {'status': 'rejected by reviewer', 'reason': 'recipient is not the customer'} {'error': 'tool delete_record is not allowed'} 4 audit entries; ['auto', 'approve', 'approve', 'forbidden']

Notice the three reviewer responses. Approve runs the action as proposed; edit corrects it (a typo turned a $49 refund into $490); reject blocks it and tells the model why, so it can continue or explain to the user. The read-only lookup never bothered the reviewer, and the unknown tool was refused without asking anyone.

For a quick command-line version, the approver can read from the terminal:

Terminal approver

def terminal_reviewer(name: str, args: dict) -> dict:
    answer = input(f"Allow {name}({json.dumps(args)})? [y/N] ").strip().lower()
    return {"action": "approve"} if answer == "y" else {"action": "reject", "reason": "declined in terminal"}

Show reviewers what they need

An approval request is only as good as the information in it. A reviewer who sees issue_refund({"order_id": "A-1001", "amount": 490}) without context will approve on autopilot. Include:

The action and arguments, in plain language (“Refund $490.00 to the card ending 4242”).
The reason the agent gives, and the user request that led to it.
Relevant facts fetched by code, not by the model (“Order total: $49.00”).
The options: approve, edit, reject with a reason.

Dry runs

Definition · Dry run

A mode where the agent plans and “executes” actions, but write tools only report what they would do. Use it to test new agents and prompts against real data safely.

Python

def execute_dry(name: str, args: dict) -> dict:
    if POLICY.get(name) == "auto":
        return IMPLEMENTATIONS[name](**args)                # reads are safe to run
    return {"dry_run": True, "would_call": name, "with": args}


print(execute_dry("get_order_status", {"order_id": "A-1001"}))
print(execute_dry("issue_refund", {"order_id": "A-1001", "amount": 49.0}))

{'order_id': 'A-1001', 'status': 'delivered', 'paid': 49.0} {'dry_run': True, 'would_call': 'issue_refund', 'with': {'order_id': 'A-1001', 'amount': 49.0}}

Pausing and resuming

Real approvals take minutes or hours, so the agent cannot sit in a blocking input() call. Instead it saves its state, the request goes to a reviewer, and the run resumes when a decision arrives — possibly in a different process.

Python

from pathlib import Path

PENDING = Path("pending")
PENDING.mkdir(exist_ok=True)


def pause_for_approval(run_id: str, messages: list[dict], name: str, args: dict) -> None:
    state = {"run_id": run_id, "messages": messages, "pending": {"tool": name, "args": args}}
    (PENDING / f"{run_id}.json").write_text(json.dumps(state), encoding="utf-8")


def resume(run_id: str, decision: dict) -> list[dict]:
    path = PENDING / f"{run_id}.json"
    state = json.loads(path.read_text(encoding="utf-8"))
    call = state["pending"]
    if decision["action"] == "reject":
        result = {"status": "rejected by reviewer", "reason": decision.get("reason", "")}
    else:
        result = IMPLEMENTATIONS[call["tool"]](**decision.get("args", call["args"]))
    state["messages"].append({"role": "tool", "content": json.dumps(result)})
    path.unlink()                                           # the approval is used exactly once
    return state["messages"]                                # hand back to the agent loop


pause_for_approval("run-7", [{"role": "user", "content": "Refund order A-1001"}],
                   "issue_refund", {"order_id": "A-1001", "amount": 49.0})
print("waiting:", sorted(p.name for p in PENDING.iterdir()))
messages = resume("run-7", {"action": "approve"})
print(messages[-1]["content"])
print("waiting:", sorted(p.name for p in PENDING.iterdir()))

waiting: ['run-7.json'] {"refunded": 49.0, "order_id": "A-1001"} waiting: []

Agent frameworks such as LangGraph and the OpenAI Agents SDK provide this as “interrupts” or “approvals”, persisting the state for you. The idea is exactly the same.

Try it

Add a rule: refunds under $20 are auto-approved, larger ones need a reviewer.
Write AUDIT_LOG to an append-only JSONL file and print a daily summary of approvals and rejections.
Make approvals expire: if no decision arrives within an hour, resume treats it as a rejection.

Key takeaways

Classify every tool as read-only, reversible, approval-required, or forbidden.
Enforce approval in the tool-execution code — never only in the prompt.
Reviewers need approve, edit, and reject, plus the context to decide well.
Log every action with its arguments, policy, decision, and result.
Long approvals need pause-and-resume: persist the run state and continue later.