Autonomous agent governance starts with a simple problem: an agent that books travel, answers customers, or moves money leaves a trail of decisions no human watched it make. When something goes wrong — a wrong refund, a leaked document, a payment nobody approved — the enterprise needs to reconstruct what happened, why it happened, and who is accountable. This article sets out the governance policies that make autonomous agent actions auditable: what to record, who may do what, who reviews it, and how the policies map to the frameworks auditors check.
The stakes are not theoretical. A 2026 Gravitee survey of more than 900 executives and practitioners found that 88% of organisations reported confirmed or suspected AI agent security incidents in the past twelve months — while only 21.9% treat agents as independent, identity-bearing entities. The policies below close that gap.
What auditing an autonomous agent actually means
An agent’s action is a chain of observable records: a goal it was given, a declared plan, the tool calls it made, and the data it read and wrote. That chain is what an audit reconstructs. Traditional application logs record that a request arrived and a response left; they were built for software whose behaviour is largely repeatable — and request/response records alone cannot capture an agent’s planning and tool-call chain. An agent is different in three ways. It plans non-deterministically, so the same instruction can produce different tool-call sequences on different runs. Its work is multi-step, so a single user request can fan out into dozens of tool calls across several systems. And its actions are tool-mediated, so the interesting events happen in other systems — a CRM, a payment provider, a mailbox — not inside the agent itself.
Internal reasoning traces are a separate, deployment-dependent record, and they do not solve this problem on their own: a chain of thought can be a plausible post-hoc narrative rather than a faithful trace of computation. OpenAI has hidden the raw chain of thought since its first reasoning model in 2024, showing users a summary instead, and in a 2025 Anthropic test, models that used a planted hint mentioned it only 25% of the time (Claude 3.7 Sonnet) and 39% (DeepSeek R1). The audit baseline must therefore work from observable records — declared plan, decision summary, tool calls — and treat an internal reasoning trace, where the platform exposes one, as supporting evidence, never as the sole explanation of an action.
Every policy in this article answers one of three audit questions: what happened, why did it happen, and who is accountable. If a policy does not help answer one of those three, it is decoration.
The audit trail: what must be recorded
The audit trail is the foundation everything else stands on. Agent audit logs are immutable, time-stamped records of every action: the model requests issued, the external tools called, the arguments passed, the results received, and the identity that authorized each operation. They are built for forensic reconstruction and compliance evidence, and they must survive independently of the systems that generated them — an agent must never be able to write to, edit, or delete its own audit log.
What goes into a record matters as much as the fact that one exists. A complete tool-call log entry captures the action type, the tool, the arguments, the result, the timestamp, the session identifier, the outcome (success, failure, rate-limited, or denied), the credential reference, and the user identity behind the call — plus the HTTP status code for HTTP-backed calls. Denied calls are logged in the same structured format as successful ones: a successful tool call proves what the agent did, and a denied call proves that the controls worked.
Observability and audit are related but not the same thing. Agent observability has five layers, each with a distinct consumer: the trace (per-request execution path, for engineering debugging), the tool-call log (every tool invocation and result, for governance audit), the memory-delta (writes to persistent and shared memory), the decision-point snapshot (inputs and outputs at significant decisions, for the regulator or auditor), and the audit record — the immutable, tamper-evident subset that survives legal and regulatory scrutiny. Not every observability signal is audit-quality, and not every audit record is useful for real-time debugging.
A practical audit record looks like this:
{
"event_id": "evt_9f31ac22",
"timestamp": "2026-11-08T14:02:11Z",
"agent_id": "agent:claims-payments-01",
"user_identity": "user:underwriter-118",
"credential_ref": "cred:svc-claims-pay-7f",
"session_id": "ses_77c1d9",
"goal": "goal:ses_77c1d9/pay-claim-2041",
"declared_plan": "plan:ses_77c1d9/02",
"decision_summary": "summary:ses_77c1d9/04",
"action": "tool_call",
"tool": "claims.issue_payment",
"arguments": {"claim_id": "CLM-2041", "amount": 480.00, "currency": "EUR"},
"http_status": 200,
"result": "approved",
"policy_decision": "allow",
"approver": "human:underwriter-118"
}The goal, declared plan and decision summary are linked to the action through the shared session identifier, so an auditor can follow one session from its declared plan to every tool call it produced.
Two policies govern what enters that record. First, data classification, minimization and redaction: distinguish required action metadata from payload content. The metadata — action, tool, agent identity, timestamps, parameters — is always recorded. Payload content is recorded only after classification: sensitive content is minimized or redacted before it enters the trail, so the log stays complete where the law and auditors need it while sensitive data never lands in logs. Second, authorized log access and retention/deletion handling: name who may read the trail, under what process, and how long records are kept — with exceptions that preserve an auditable record of what was withheld and why. Retention periods, tamper-evidence (append-only storage, hash chaining), and separation of the audit log from the agent’s own writable storage complete the baseline.
Access and permission policies for agents
Auditing is far weaker without boundaries on what an agent may do at all. Access policy has three parts. First, least-privilege tool and data access, scoped per agent role and task: an agent that triages claims does not get the payment tool, and an agent that reads a CRM does not get the deletion API. Second, identity for agents: service identities distinct from human users, with their own credentials and revocation. The Gravitee finding that only 21.9% of organisations treat agents as identity-bearing entities is the gap this policy closes — shared credentials undermine reliable per-agent attribution. Third, approval gates for high-impact actions — payments, external messages, deletions — that pause execution and require a human decision before the action runs.
The strongest implementations make these policies deterministic rather than advisory. Amazon Bedrock AgentCore, for example, structures agent operations into four pillars — governance and security, build and operations, evaluation, and observability — and its AgentCore Policy component is a deterministic engine built on the Cedar policy language that intercepts tool requests passing through the Gateway and evaluates them against business rules before execution, using the caller’s identity claims, the tool name and the tool input; its modes are LOG_ONLY or ENFORCE, the default is deny, and forbid wins. AgentCore Identity manages agent authentication and authorization, AgentCore Gateway centralizes tool access for calls routed through it, and AWS CloudTrail provides the full audit trail. The pattern generalizes: route every tool call through a governed endpoint, evaluate policy before execution, and log the decision either way.
Human oversight and review policies
Logging without review is write-only storage. Human oversight policy names who watches the audit trail, how often, and with what authority. The EU AI Act’s Article 14 states the standard clearly for high-risk systems: they must be designed so natural persons can effectively oversee them — understanding capabilities and limits, remaining aware of automation bias, correctly interpreting outputs, deciding not to use or override outputs, and intervening or stopping the system. Article 26 puts the organisational duty on deployers: assign human oversight to natural persons with the necessary competence, training and authority. Naming a role without competence or authority does not meet the duty.
In practice, oversight means three things. Human-in-the-loop checkpoints for irreversible or high-risk actions, built into the flow. Scheduled audit reviews, with sampling rates and a named owner who signs off: privileged agent access reviewed on a cadence (ISO 27001 requires privileged access rights to be reviewed at minimum every six months), and the review record itself kept as evidence. And escalation paths for when a review finds an anomalous or non-compliant action, so a finding reaches someone who can stop the agent, not just someone who can file it.
Anomaly detection and incident response
Audit data earns its keep when it is used actively, not just stored for regulators. Anomaly detection starts by baselining normal agent behaviour — typical tool use, spend, data access, working hours — and flagging deviations. Automated alerts on policy violations should fire from the same trail the auditors read, with the audit record attached as the evidence bundle. Frameworks such as Hugging Face’s smolagents now ship structured logging of agent actions, tool calls, reasoning steps and resource access, including logging of permission-boundary violations — attempts to act outside defined boundaries are recorded with the same detail as permitted actions, which is exactly what an anomaly detector needs.
The incident workflow is the other half: freeze the agent’s access, reconstruct the sequence from the trail, remediate, and feed the lesson back into policy. If the trail holds declared plans, decision summaries and tool calls with identity attached, reconstruction can draw on the full sequence of what the agent did and why; if it holds only outcomes, the record may never answer why. Every incident review should assess and document the policy implications, and change permissions, approval gates or baselines when the findings justify it.
Mapping autonomous agent governance policies to compliance obligations
Internal policies become defensible when they map to frameworks auditors already check. Three frameworks dominate enterprise audits. The EU AI Act, Regulation (EU) 2024/1689, requires high-risk AI systems to technically allow the automatic recording of events over the system’s lifetime (Article 12), to be designed for effective human oversight (Article 14), and deployers to keep logs under their control for at least six months unless other EU or national law provides otherwise (Article 26(6)). The high-risk obligations apply from 2 December 2027 for Annex III systems and 2 August 2028 for Annex I products, following the Digital Omnibus. Critically, these duties apply to high-risk systems — a general-purpose agent in an enterprise workflow is not automatically in scope, and the mapping must distinguish applicability from aspiration.
SOC 2 requires security events to be logged (CC7.2) and anomalies detected and investigated (CC7.3), but specifies no explicit retention period; access is reviewed periodically under CC6.2 and CC6.3, with annual reviews as the minimum acceptable evidence. A SOC 2 Type II report requires demonstrating that controls operated effectively across the observation window, typically six to twelve months — so the audit trail must exist and be reviewable for that whole period. ISO 27001:2022 requires logging of user activities, exceptions, faults and security events (A.8.15) with logs protected from tampering, and privileged access rights reviewed at minimum every six months (A.8.2).
When an auditor asks for evidence, they ask for specific artifacts: the last access review, who reviewed it, when it was completed, and what access was removed as a result; the dated review reports with named sign-off; and the log records showing the control operated. Review quality is judged by documented examination, named sign-off and justified outcomes: access is removed when inappropriate access is identified, but zero removals on their own do not prove a review was a rubber stamp. The policy set itself is auditable evidence too: document it, version it, and keep the version that was in force when each action was taken.
Common mistakes when auditing autonomous agents
The most common failure is logging outcomes but not the declared plan, decision summary, or tool calls, leaving actions unexplained — and, conversely, treating internal reasoning traces as a required record when many deployments cannot provide them. A trace you cannot obtain is not a control; the baseline must work from observable records, with reasoning traces as optional supporting evidence where the platform exposes them.
The second failure is letting agents write to their own audit logs or share human credentials. An audit trail the subject can edit is not an audit trail, and a shared credential makes it impossible to attribute an action reliably to one agent. Give every agent its own identity, keep the log in storage the agent cannot write to, and log denied calls as carefully as successful ones.
The third failure is reviewing logs only after an incident instead of on a schedule, and treating policy as a one-time document. A policy set that is never reviewed rots as agents gain tools and change scope. The enterprises that audit agents well share one habit: they treat the audit trail as a product — versioned, reviewed, and tested by reconstructing a real incident from it before a real incident forces them to.
