Building AI agents that cannot run away: caps, scopes and audit logs

An agent is a language model given tools and a goal, allowed to decide the sequence of steps itself. That is genuinely useful for work with a variable path, triaging an inbox, reconciling records between two systems, researching and drafting, and it is the reason agents are interesting rather than just another API call.

It is also why they are risky. A loop that decides its own next action, holding credentials that can change data, with no ceiling on iterations, is a system whose worst case you have not bounded. The engineering that makes agents deployable is almost entirely about bounding that worst case.

First: should this be an agent at all?

This is the question worth spending real time on, because the answer is often no.

If the steps are the same every time, you want a workflow, not an agent. A deterministic pipeline is cheaper, faster, easier to test, and it fails in ways you can predict. Handing a fixed sequence to a model that re-derives it on every run adds cost and variance for nothing.

An agent earns its place when the path genuinely varies, when the next step depends on what the previous one found, in ways you cannot enumerate in advance. That is a narrower category than the current enthusiasm suggests. In most workflows I automate, the majority of steps need no model at all, and identifying that first shrinks both the problem and the running cost considerably.

Cap the loop, in code, not in the prompt

Every agent loop needs a hard iteration limit enforced by the code that runs it. Not a request in the system prompt; a counter.

const MAX_STEPS = 12;
const MAX_TOKENS = 200_000;

let steps = 0;
let tokensUsed = 0;

while (steps < MAX_STEPS && tokensUsed < MAX_TOKENS) {
  steps++;
  const response = await model.run(conversation);
  tokensUsed += response.usage.input_tokens + response.usage.output_tokens;

  if (response.stop_reason !== "tool_use") return finish(response);
  conversation.push(await executeTools(response.tool_calls));
}

// Reaching here is a failure state, and should be treated as one
throw new AgentBudgetExceeded({ steps, tokensUsed });

The distinction matters. A prompt instruction is a request the model may or may not honour, and it certainly will not honour it if it has got stuck in a repetitive loop, which is exactly the situation the limit exists for. Prompt instructions are a supplement to enforcement, never the mechanism.

The same applies to spend. A daily cap checked before each call, enforced in code, not watched on a dashboard. Dashboards tell you about the bill after it happened.

Scope the tools so the bad action is impossible

This is the single highest-value principle in agent design, and it is not an AI principle at all. It is ordinary least privilege.

The agent should not be asked not to delete records. It should be holding credentials that cannot delete records.

A useful test when designing a tool: if a malicious actor controlled the model's output entirely, what is the worst this tool lets them do? If the answer is unacceptable, the tool is wrong, not the prompt. This matters more than usual with agents, because agents read external content, and external content can contain instructions. Prompt injection is not hypothetical; scoping is what makes it survivable.

An approval gate before anything irreversible

Divide the agent's actions into reversible and irreversible. Reading, drafting, classifying, staging, reversible. Sending an email to a customer, issuing a refund, deleting a file, posting publicly, not.

Irreversible actions go through a human. But the approval has to be designed to be meaningful, and this is where most implementations fail. If the reviewer sees "Agent wants to send email. Approve / Reject" fifty times a day, they will approve reflexively within a week, and you have a rubber stamp rather than a control.

What makes an approval real:

• The full content of what will be sent or changed, not a summary of it.
• Why the agent reached this conclusion, including what it retrieved.
• What it will do next if approved.
• A rejection path that captures the reason, so the failure feeds back into improving the system.

If approvals are running at high volume, that is a signal the scope is too broad. Narrow what the agent handles autonomously rather than accepting the rubber stamp.

Trace everything, because you will need it

When an agent does something surprising, and it will. You need to be able to reconstruct exactly what happened. Not approximately. Exactly.

Log per run: the full prompt including retrieved context, every tool call with its arguments, every tool result, the model's reasoning where the API exposes it, token counts and cost per step, duration, and the final outcome.

{
  "run_id": "run_01H8X...",
  "started_at": "2026-07-29T09:14:22Z",
  "trigger": { "type": "webhook", "source": "helpdesk", "ref": "TKT-4821" },
  "steps": [
    {
      "n": 1,
      "tool": "search_knowledge_base",
      "args": { "query": "refund policy for annual plans" },
      "result_summary": "3 passages, top score 0.81",
      "tokens": { "in": 2418, "out": 96 },
      "ms": 1240
    }
  ],
  "outcome": "escalated_to_human",
  "reason": "retrieval_confidence_below_threshold",
  "cost_usd": 0.043
}

Storage is cheap and this is not optional. Without it, debugging an agent is guesswork, and any question about what it did on a particular day is unanswerable.

Earn autonomy, do not grant it

New agents run in shadow mode: they do the work and propose the action, a person reviews and executes. Nothing goes out automatically.

Run that for an agreed period. A couple of weeks, or a set number of cases, and measure the agreement rate. If the human approves 95% of proposals without modification, autonomy on that class of action is a reasonable next step. If they are editing half of them, you have learned something valuable cheaply, and no customer was involved in the learning.

Expanding autonomy incrementally, per action type, from an observed track record, is slower than launching fully autonomous. It is also the difference between a system people trust and one they quietly stop using.

The failure modes worth designing for

• The repetition loop. The agent calls the same tool with the same arguments repeatedly. Detect duplicate consecutive calls and break out rather than waiting for the iteration cap.
• Confident wrongness. The model states something false with no hedging. Mitigated by grounding answers in retrieved sources and requiring citations, not by asking for accuracy.
• Prompt injection from retrieved content. A document or email containing instructions the model follows. Mitigated by tool scoping, if the injected instruction asks for something the credentials cannot do, it fails harmlessly.
• Silent degradation. Quality drifts after a model or prompt change and nobody notices. Mitigated by an evaluation set that runs on every change, scored automatically.
• Provider outage. The model API returns 500s. The agent should fail into a queue and a human notification, not into a partially completed workflow.

The short version

Prefer a deterministic workflow unless the path genuinely varies. Cap iterations and spend in code. Scope tools so the dangerous action is impossible rather than discouraged. Put a real human decision in front of anything irreversible. Trace every run completely. Earn autonomy from evidence.

None of this is about the model. It is ordinary systems engineering applied to a component that is non-deterministic, and it is what separates something you can put in front of a business from something that demos well. More on how I approach this under AI agents and workflow automation.

Written by Irfan Bhat, web developer & server engineer, currently technical support & development at ElySpace. More about me, or get in touch if this raised a question.