RYZ LABS / Blog

← Back to Blog

November 16, 2026

Human in the loop AI agents: where people must approve

Human in the loop AI agents, designed well: approval tiers by risk and reversibility, escalation triggers, review queues, override logs and review SLAs.

Human in the loop AI agents: where people must approve

Human in the loop AI agents work when people approve only the actions that are hard to undo or costly to get wrong, and the agent runs on its own everywhere else. Sort every tool an agent can call by risk and reversibility, put a checkpoint where both are high, and give reviewers a queue with enough context to say no. Then log every override, set a review deadline, and remove checkpoints only when the data says they no longer catch anything.

Why human in the loop AI agents became an engineering question

A drafting agent that writes an email for a person to send has a natural human checkpoint: the send button. An agent that sends the email itself, updates the CRM record, or schedules the vendor payment does not. The checkpoint now has to be designed, built, staffed and measured, which makes it an engineering and operations problem, not a policy memo.

Year-end close and Q1 planning are when leaders ask agents to do more, and when a wrong payment costs the most. Decide where humans approve before the first tool is wired up.

Regulators and standards bodies already describe what good oversight looks like. The EU AI Act, Article 14 requires that high-risk AI systems can be effectively overseen by people while in use, including the ability to disregard, override or reverse an output and to halt the system safely through a stop button or similar procedure. It also names automation bias, the tendency to over-rely on a system's output, as a risk overseers must watch. The NIST AI Risk Management Framework asks organizations to define and separate human and AI roles (GOVERN 3.2), document human oversight processes (MAP 3.5), and keep mechanisms to supersede, disengage or deactivate a system that behaves outside its intended use (MANAGE 2.4). If you want a plain definition first, see our human-in-the-loop glossary entry.

Next question: Who builds approval steps into an agent that runs on your own systems?

Sort every action by risk and reversibility

Decide oversight per action, not per agent. The same agent might read a contract, update a deal stage and issue a refund. List every tool it can call and score each on two questions: how bad is a wrong call, and how easily can it be undone?

Coding tools already work this way. In Claude Code's permission system, read-only actions such as file reads run without approval inside the working directory, while shell commands (outside a built-in read-only set) and file edits ask first. The OpenAI Agents SDK lets developers mark a tool with needs_approval, either always or through a function that decides per call from the tool's arguments. Approval is a property of the action and its parameters.

A starting map to adjust to your systems:

Action typeRisk if wrongReversible?Recommended human checkpoint
Read records, search documents, summarizeLowNothing to reverseNone; log the run
Draft an email, note or reply for a personLowYesNone; the person sending is the checkpoint
Update internal fields (CRM stage, ticket tags)ModerateYes, with an audit trailSampled review after the fact
Send an external message to a customer or vendorModerate to highNoApprove before send until quality is proven, then sample
Change customer data, contracts or entitlementsHighPartlyApprove before commit, show a diff
Pay an invoice, issue a refund, move moneyHighRarelyApprove every action above a set amount; two people above a higher one
Delete data, change permissions, deploy codeHighOften notAlways approve; restrict who can approve

Approval does not replace permissions. If an agent should never delete records, remove the delete tool rather than relying on a reviewer to catch it. We cover that side in our post on agent permissions and prompt injection.

Next question: What do AI agent developers cost per month?

Escalation triggers: policy, confidence and novelty

Static tiers are the floor. Add triggers that escalate an action that would normally run on its own.

  • Policy triggers. Hard rules written as code: amount over a threshold, a new payee, a customer flagged as strategic, a regulated data field, an action outside business hours. These are cheap, explainable and easy to audit.
  • Confidence triggers. The agent's own uncertainty, a classifier score, or disagreement between two models. Treat these with care: a model's stated confidence is often poorly calibrated, so measure whether low-confidence cases really are wrong more often before you trust the signal.
  • Novelty triggers. Inputs unlike anything in your test set: a new document type, an unfamiliar vendor, a language the agent rarely sees. Novel cases are where agents fail quietly.

Write triggers as configuration operations can read, not prose buried in a prompt, so anyone can point at the rule that let a payment through.

Next question: How do you measure whether an agent's confidence score means anything?

Build a review queue reviewers will actually use

Most approval flows fail at the screen, not the model. A pop-up that says "Approve this action?" with a JSON blob underneath teaches people to click yes. A useful review item shows:

  1. The proposed action in plain language, with the exact parameters (payee, amount, recipient, fields changed).
  2. A diff for any change to a record or document, so the reviewer sees before and after.
  3. The evidence the agent used: the source invoice, the email thread, the policy clause, linked rather than pasted.
  4. Why it was escalated: which rule or trigger fired.
  5. Four choices, not two: approve, reject with a reason, edit and approve, or send back to the agent with a note.

Put the queue where reviewers already work, such as the ticketing tool or finance system, and route by skill. Treat a pending approval as durable state: the OpenAI Agents SDK pauses the run and lets developers serialize its state to resume later, and its docs warn the stored snapshot is not authenticated, so keep it on trusted server-side storage. A reviewer who answers tomorrow should resume the same run.

Next question: Who builds reviewer tools alongside your operations team?

Rubber-stamp fatigue is the real failure mode

A checkpoint that approves everything is worse than no checkpoint, because it gives everyone false comfort. Article 14's warning about automation bias is a design requirement, not a footnote. Watch for these signs:

  • Approval rates that sit near total for weeks.
  • Median time on a review item measured in a second or two.
  • Reviewers approving in bulk at the end of the day.
  • Rejections that cluster on one reviewer while others never reject.

Cut volume first: a tier with many approvals and no rejections is a candidate for sampling. Batch similar low-risk items and rotate reviewers on high-stakes queues. Most useful of all, seed the queue with known bad items and track how many reviewers catch, as part of your agent testing. That makes reviewer attention measurable.

Next question: How do you test that reviewers catch the errors you plant?

Log every override as eval data

Each rejection or edit is a labeled example of the agent being wrong, written by the person best placed to judge. Store it with the full context: input, proposed action, reviewer decision, the edit, the stated reason, and the agent and prompt version. NIST's framework names appeal and override as part of post-deployment monitoring (MANAGE 4.1), and this is the practical version of that.

Group rejections by reason each week, turn recurring ones into eval cases, and require prompt or model changes to pass them before release. The override log becomes the most realistic test set you own. Human judgment stays central even at volume: an AI fraud detection system our pods shipped for a global fleet company has scored more than 244,000 invoices, with results validated by the client's own fraud team.

Next question: Who turns reviewer overrides into an eval suite?

Review SLAs, and what happens when nobody answers

An agent waiting on a person is a queue, and queues need service levels. Set a review target per tier from the business process: a customer reply may need an answer within the hour, a vendor payment may have until the next business day.

Decide timeout behavior in advance. For high-risk actions, do nothing and escalate to a named backup, never auto-approve. For low-risk items escalated by a soft trigger, you might proceed after the deadline and flag for later review. Show the clock on each item and report SLA misses with agent errors. Plan for holidays: a Thanksgiving-week backlog gets approved in a hurry on Monday.

Next question: Who runs an agent and its review queue after launch?

When to remove a checkpoint, and who owns it

Agree exit criteria when you add a checkpoint. Before moving an action from "approve first" to "sample after", check:

  • The action has run through review long enough to cover your normal monthly cycle, including month-end.
  • Rejections and edits have dropped to a level the business owner accepts, and the remaining ones are minor.
  • Planted bad items are still being caught, so the low rejection rate is not just fatigue.
  • The recurring rejection reasons are covered by eval cases that pass.
  • Policy triggers stay on, so the highest-risk variants still escalate.
  • Rollback is tested: someone can stop the agent and reverse the last actions quickly.
  • The decision, the data behind it and the date to re-check are written down.

Split ownership, as GOVERN 3.2 asks. The process owner (the controller for payments, the support lead for messages) owns the risk tier and policy. An engineering lead owns triggers, queue, logs and evals. A named reviewer group owns the SLA.

Next question: What does a senior AI engineer cost per month?

How Ryz can help

Ryz AI pod teams are senior engineers who work on your team, in your cloud and repos, during US business hours, and build agents that act inside your CRM, finance and support systems with the approval tiers, review queues and override logs described here. Only the top 1% make it through our vetting, and most of our engineers ship code in their first week. See our AI agent development work, or hire AI agent developers to join your team.

FAQ

What is human in the loop for AI agents?

It is a design where a person reviews, approves, edits or rejects certain agent actions before or after they happen. For agents that take actions, the useful version is selective: approval on high-risk, hard-to-reverse actions, and logging or sampling everywhere else.

Which agent actions should always need human approval?

Actions that move money, delete data, change permissions, deploy code, or change contracts and customer entitlements. Many teams also require approval for external messages until the agent has a proven record, then move to sampling.

Does the EU AI Act require human in the loop?

Article 14 requires that high-risk AI systems can be effectively overseen by people, including the ability to override or reverse outputs and to stop the system safely. Whether your agent counts as high-risk depends on its use, so check with your legal team.

How do you stop reviewers from rubber-stamping agent actions?

Reduce review volume to the actions that matter, show clear context and diffs, seed the queue with known bad items to measure catch rates, rotate reviewers, and watch approval rates and time spent per item.

Work with Ryz Labs

Senior engineers on your team and AI pod teams that ship to production, on US business hours. Typical cost is $7,000 to $15,000 per engineer per month.

Come build with us

Let's buildsomething great,together.

Tell us what you are building. You will get a scoped plan, a price, and the names of the people who would do the work.