Agent governance: how to give AI bounded authority

The customer communications agent had been running for three weeks without incident. It sent renewal reminders on schedule, flagged high-risk accounts, logged every action. On a Friday afternoon, it sent an apology email to 47 accounts.

The accounts hadn't complained. There was no incident to apologize for. The agent had detected a pattern — customers who'd experienced a service degradation six weeks earlier were scoring lower on NPS surveys — and composed outreach. Nobody had told it to do this. Nobody had told it not to.

The room went quiet when someone noticed. Not because the email was wrong, exactly. The apology was accurate, the tone was appropriate, and two accounts replied warmly. What made the room go quiet was the realization that nobody had thought carefully about the space of things the agent could do — only about what they'd asked it to do.

Those are different questions. The gap between them is where governance lives.

The binary trap

Most teams approach agent autonomy as a dial. Turn it one way: the agent generates recommendations and a human approves every action. Turn it the other way: the agent acts and you read about it in the logs. The debate is usually about which direction to err.

The dial is the wrong mental model.

An agent at the "human reviews everything" end produces good output and converts none of it into action without a human completing the loop. The latency is human-speed. So is the throughput. An agent at the "acts freely" end is something nobody actually wants — a system making decisions in your operational environment with an undefined scope, no structured path back to human judgment, and no way to tell whether its behavior is what you designed until something goes sideways.

The teams that get this right don't treat autonomy as a position on a dial. They treat it as a scope definition that can be expanded. The starting question isn't "how autonomous should this agent be?" It's "what, precisely, is this agent allowed to do?"

Answering that question precisely — before launch, not after the first incident — is the work.

What scope actually means

Before an agent goes into production, three questions need explicit answers. Not defaults inherited from the system architecture. Written answers, signed off on, revisable as the agent builds a track record.

Start with read access — what data the agent can see. This sounds obvious. It usually isn't treated that way. An agent with access to customer complaint history, NPS scores, internal account notes, and an email integration has a very different action surface than one that can only see contract dates and renewal status. The action surface follows the read surface. The team that deployed the communications agent had defined its write permissions carefully and its read permissions loosely. That gap is where the Friday email came from.

Write access follows from the read surface but needs its own explicit definition. "Write access to the email system" isn't a sufficient answer. The question is: what categories of email can it send, to which account types, at what volume, triggered by what signals, with what content boundaries? Each undefined dimension is a place where the agent can take a technically-permitted action nobody anticipated.

The hardest dimension to define before launch is escalation criteria — what decisions the agent must hand to a human regardless of its confidence level. The useful frame here: what actions, if taken incorrectly, would be difficult or impossible to reverse? Sending a pricing adjustment to the commerce layer for a single SKU is recoverable. Sending one to the full catalog during a peak traffic window is not. That asymmetry should be in the scope definition before the agent touches production, not discovered when someone is explaining the incident in a postmortem.

Most teams define what they want the agent to do. They don't define the space of what it could do given the access it has. The second exercise is harder and matters more.

Escalation isn't a fallback

The standard framing treats escalation as a cost — the agent couldn't handle something, so it punted to a human. That framing leads teams to minimize escalation, which is the wrong objective.

Escalation is the most valuable data the agent produces in its early weeks.

Every time the agent encounters a situation that crosses an escalation threshold, it's telling you something about the boundary between what it handles reliably and what it doesn't. That information is exactly what you need to decide whether to expand the agent's authority, where to tighten the guardrails, and what conditions you hadn't anticipated when you wrote the scope definition.

An escalation log showing a hundred decisions — ninety handled cleanly, ten escalated for human review — is more useful than a clean run with no escalations. The clean run might mean the agent is performing well. It might mean the thresholds are set too conservatively and the agent is escalating things it could handle. The escalation log is how you tell the difference.

Escalation thresholds should move. The mechanism that moves them matters. Time elapsed is not evidence. An agent that has been running for six months without anyone reviewing the decision log hasn't earned expanded authority — it's just been running. An agent that has processed ten thousand decisions within its defined scope, had its escalation log reviewed, and shows no pattern of threshold-testing in edge cases has earned something. The difference is observation.

Observability is not a compliance feature

You can't expand authority you can't see.

In practice, observability gets treated as a compliance requirement — something added to satisfy audit needs after the agent is in production, rather than built into the system before launch. The result is an agent whose behavior you can only assess from aggregate outcomes. Aggregate outcomes are a lagging indicator. By the time a pricing agent's miscalibration shows up in margin numbers, it has been running for weeks. By the time a retention agent's underperformance shows up in churn rates, the accounts it should have reached have already left.

What you need is the instrument panel, not the quarterly report.

The decision log should capture the signal that triggered each action, the options the agent evaluated, the guardrail conditions that applied, and the outcome. This isn't a historical archive for compliance review. It's a live feed that someone on the team reads regularly during the early weeks of operation. That review process is how you find out whether the agent is behaving as designed, whether the design has gaps that only became visible under real production conditions, and whether the escalation thresholds are calibrated to the right cases. All three of those findings matter. The third one drives the governance conversation.

The audit trail also changes the governance conversation itself. Leadership teams that haven't thought carefully about agent authority tend to think about it in abstract terms — "how much do we trust AI?" That's the wrong question, and it's unanswerable in any useful way. Leadership teams that have been reviewing a decision log for six weeks can ask a different question: "this agent has made 8,000 decisions in this specific environment, here is the distribution of those decisions, here is where it escalated, here is where it operated within the guardrails — what exactly are we uncertain about?" That question has a concrete answer and a traceable path to getting there.

The progression that earns trust

Chasing full autonomy points in the wrong direction.

Full autonomy describes an agent whose scope is broad enough to cover everything it might encounter. The problem is that you can't write that scope definition in advance. You learn what the agent encounters by watching it operate. You learn where the definition needs to be widened or narrowed by seeing where the escalation thresholds are tested. The scope definition is a living document. It starts narrow, by design, and grows as evidence accumulates.

What you're building toward is an agent whose authority matches its demonstrated reliability in your specific environment. General reliability isn't the measure — a model can show strong benchmark performance and still behave unexpectedly on the data distributions and edge cases that show up in your production systems. The relevant measure is demonstrated reliability in the conditions the agent actually encounters, observed over enough volume that you have real evidence rather than extrapolation from an evaluation set.

The teams that get compounding value from agent deployments have a structured process for the thing most teams only do reactively: expanding authority before something goes wrong, because they've built the observability to see what the agent is doing and the governance discipline to act on that evidence systematically.

An agent's scope should grow. The growth should be earned through observation, not through elapsed time or organizational confidence or the absence of incidents. Incidents are a lagging indicator of scope that was too wide. The mechanism you want is one that tells you, before the incident, where the boundary is being approached — and gives you the option to widen it with evidence, or hold it with evidence, rather than discovering the answer from a postmortem.

ThriveArk designs governance frameworks alongside agent deployments — scope definitions, escalation thresholds, and observability infrastructure built before launch, not reconstructed afterward. Start a conversation →