AI Agents, Evergreen

AI Agent Security: What to Check Before You Deploy

A cheerful robot eagerly accepts a sealed envelope with a fishing hook inside from a masked figure while a worried human holds up a stop hand. Headline: Your agent trusts everything.

AI agent security is the set of controls that keep an autonomous system from doing something its operators would not approve of. The moment an agent can read documents, call tools, and change records, a prompt problem stops being a chatbot problem and becomes a systems problem. This guide covers the security considerations that matter when deploying AI agents: identity and permissions, tool and data controls, prompt injection, data leaks, monitoring, supply-chain risk, and a pre-launch checklist you can run through before go-live.

How deploying an AI agent changes your security surface

A conventional service runs code its developers wrote. An agent runs instructions assembled at runtime from prompts, retrieved documents, and tool outputs — sources that may not be trustworthy. That is the core change: the agent takes direction from content it reads, not only from code you shipped.

Three properties widen the attack surface. First, autonomy: a single bad decision can chain into many real-world actions before a human notices. Second, tool use: the agent holds API keys and credentials that can send messages, move money, or delete records. Third, persistent memory and retrieval: what the agent learned in one conversation can surface in another. OWASP’s companion Top 10 for Agentic Applications was built for exactly this situation, listing risks specific to systems where AI plans and takes actions with tools. Its first entry, ASI01 Agent Goal Hijack, is the agentic form of prompt injection.

The practical consequence is that deployment security for agents is not a hardening checklist for a server. It is a control problem for a system that reads untrusted text and acts on it.

AI agent security starts with identity and least privilege

Give the agent its own identity. An agent should never run on a shared human credential, because then its actions are indistinguishable from a person’s, and revoking its access means revoking a human’s. A dedicated service identity makes the agent’s actions auditable and its access revocable on its own.

Scope that identity’s permissions to the minimum each task requires. NIST’s discussion of agent identity highlights the risks of broad local user access and static credentials, and Google’s guidance is to use trusted tools, scope permissions to the minimum required, and prefer short-lived credentials. If the agent only reads tickets, it should not also be able to refund payments or change access controls.

Separate read and write capabilities. Reads can usually run automatically; high-impact writes — sending externally, deleting data, changing access, deploying — should take a separate path with human approval, while bounded, reversible writes can proceed under explicit application-enforced policy. OWASP calls the failure mode here Excessive Agency (LLM06:2025): a model given more tools, permissions, or autonomy than the task needs can be manipulated into operations far beyond its intended scope, from reading sensitive files to executing code or modifying database records.

Controlling what the agent can touch: tools, APIs and data

Allowlist the agent’s tools and endpoints. Block direct shell, network, and filesystem access by default, and add tools back only when a task genuinely needs them. Every tool the agent can call is part of its attack surface; excluding a tool removes that direct execution path and reduces exposure — but it does not eliminate the risk, since an allowed tool or rendered output can provide another route, as the EchoLeak example shows.

Validate tool arguments at the tool boundary, before execution. Check the target resource, the permitted action, the data scope, and any limits on the change. A tool call that looks reasonable in a plan can still be dangerous with a different argument — a delete call aimed at the wrong record, or a send call aimed at an external address.

Restrict which data sources the agent can read and which records it can modify. Retrieval must respect the requesting user’s access rights: an agent answering one user must not surface another user’s records because the retrieval index has no per-user boundary. The same applies to memory — stored conversation and retrieved context must be scoped to the user who produced them.

Prompt injection and untrusted input

A robot reads a document with hidden instructions glowing red while a shadowy hand reaches from the page toward its tool belt.

Prompt injection is the leading risk in the OWASP Top 10 for LLM Applications 2025, ranked LLM01:2025 at number one. The reason it holds the top position is that there is no complete fix: a language model cannot reliably tell instructions from data, so text from a user, a document, a web page, or a tool result can redirect what it does.

Direct injection comes from the user. Indirect injection is planted in content the model later reads — and for agents, that is the dangerous form, because agents read email, tickets, web pages, and tool outputs as part of their normal work. The UK National Cyber Security Centre puts the core problem in one sentence:

Current large language models do not enforce a robust boundary between instructions and untrusted content inside a prompt.

EchoLeak is the reference case. In June 2025, Microsoft disclosed CVE-2025-32711, an AI command injection issue in Microsoft 365 Copilot. Researchers at Aim Labs showed that a single crafted email could make Copilot disclose information over a network — without the user ever opening the email or clicking anything. Copilot retrieved the planted email as context for an unrelated question and followed the hidden instructions. Microsoft patched the vulnerability server-side and confirmed no evidence of exploitation in the wild.

Defences are layered, not absolute. Keep system instructions authoritative and treat tool output and retrieved content as data, never as instructions. Validate inputs and filter outputs. And before any consequential action, require confirmation — a human or a policy check that sees exactly what the agent proposes to do.

Preventing data leaks and exfiltration

Agents handle sensitive data constantly: personal information in prompts, secrets in tool calls, internal records in retrieved context, and all of it in logs. OWASP promoted Sensitive Information Disclosure (LLM02:2025) to number two in the 2025 list, because models can reveal system prompts, API keys embedded in context, or personal information from conversation history.

Three controls form important layers of defence. First, minimize what the agent sees: pass only the fields a task needs, and redact secrets and personal data before they enter a prompt. Second, enforce per-request access control on retrieval and memory, so one user’s data never leaks into another user’s answer. Third, control egress: restrict where the agent can send data, and treat rendered output as a possible exfiltration channel — EchoLeak exfiltrated data through the assistant’s own rendered answer, not through a direct network call.

Logs deserve the same treatment. Agent logs record prompts, tool calls, and retrieved content, which means they can contain the same sensitive data the agent handled. Redact them, restrict access to them, and set retention limits.

Guardrails, monitoring and human oversight

A robot waits at three checkpoint gates: green for reading, yellow for drafts, and red where a human holds a green approval stamp.

Guardrails are automatic checks on inputs and outputs. Human review is a decision: a pause before a sensitive action where a person or policy approves or rejects it. OpenAI’s Agents SDK documents human review as pausing a run so a person can approve or reject a sensitive action, with the run resuming the same state after approval. The two are complements, not substitutes.

Put approval gates around consequential actions, and use risk tiers instead of one global prompt. Reads and calculations can run automatically. Drafts and staging changes can run with constraints. High-impact actions — sending externally, deleting data, making purchases, changing access, or deploying — need human approval of the exact action, target, and parameters. Bounded, reversible actions, such as creating a draft or updating a staging record, can proceed under explicit application-enforced policy instead of a human gate. A gate on every action does not work: Anthropic’s study of Claude Code’s permission model found users accept 93% of permission prompts, which turns approval into a rubber stamp. Anthropic’s response was an auto mode that classifies each tool call and surfaces only the ones that warrant human attention.

Log every tool call, decision, and data access with enough detail to reconstruct an incident. Add rate limits, action budgets, and anomaly alerts on agent behaviour — an agent suddenly making ten times its normal number of tool calls is a signal worth investigating. And define the failure path in advance: if review times out or is unavailable, fail closed and do not execute the action.

An example policy makes the tiers explicit. This is an illustrative configuration, not a vendor schema:

{
  "agent": "support-triage",
  "tools": {
    "allowed": ["search_tickets", "read_ticket"],
    "approval_required": ["send_reply", "refund_payment", "delete_record"],
    "denied": ["shell_exec", "database_write"]
  },
  "on_review_timeout": "fail_closed"
}

The point of writing the policy down is that it is enforced by the application, not by asking the model to behave. Enforcement belongs in the application or tool path that controls execution.

Model and supply-chain risks

Model version changes can alter agent behaviour in production. A model update can change how the agent interprets instructions, which tools it reaches for, and how it handles edge cases. Pin model versions and test before upgrading, the same way you would test a dependency upgrade.

Third-party tools, plugins, and MCP servers are supply-chain exposure. Each one introduces dependencies and trust boundaries of its own: inspect the integration’s actual execution environment, credentials, scopes, and accessible data, rather than assuming it runs with the agent’s permissions. OWASP lists Supply Chain (LLM03:2025) among the top risks and Agentic Supply Chain Vulnerabilities (ASI04) in the agentic list: compromised models, datasets, adapters, or tools. Vet what you connect, keep the list short, and review it when the agent’s role changes.

Retrieval and fine-tuning data can be poisoned too. Data used for fine-tuning or placed in a retrieval index can carry instructions or false facts that the agent later treats as ground truth. Treat the contents of your knowledge base as an input you control, and validate it the way you would validate any other data source.

The Five Eyes agencies, including CISA and the NSA, issued joint guidance on agentic AI in May 2025 that assumes prompt injection will not be solved, and recommends deploying agents incrementally with human oversight at consequential decision points. That is the right posture for the whole supply chain: assume the controls you have are imperfect, and keep a human in the loop where it matters.

AI agent deployment security checklist

Run through this list before go-live, and again whenever the agent’s tools, data sources, or model change. Test the controls, not just their presence: try retrieving another user’s record with a second identity, check that a document carrying an injected instruction cannot authorize a tool action, and confirm that an expired or missing approval prevents execution. These are suggested checks, not proof of complete security.

  • Identity: the agent has a dedicated service identity with short-lived credentials, not a shared human login.
  • Permissions: tools and API access are scoped to the minimum the job needs; destructive actions are denied or gated.
  • Approval gates: high-impact actions require human approval of the exact action, target, and parameters; bounded actions may proceed under explicit policy; the policy fails closed on review timeout.
  • Injection defences: retrieved content and tool output are treated as data, not instructions; inputs are validated and outputs filtered.
  • Data handling: retrieval and memory respect per-user access rights; secrets and personal data are redacted from prompts and logs; egress is restricted.
  • Logging and monitoring: every tool call and decision is logged with enough detail to reconstruct an incident; rate limits and anomaly alerts are in place.
  • Rollback and kill switch: you can stop the agent, revoke its credentials, and undo its recent actions — and you have rehearsed doing so.
  • Model and supply chain: the model version is pinned; third-party tools and plugins are vetted and minimal.

Security for deployed agents is not a one-time review. The agent reads new content every day, its tools change, and its model gets updated. The checklist above is the baseline; the habit of re-running it is the control that keeps working after launch.