AI agent security is mostly about what an agent is allowed to do, not what it is told to do. Assume prompt injection will get through at some point, then design so a hijacked agent cannot reach much data, cannot act outside a narrow set of scoped tools, and cannot take an irreversible action without a human approving that exact action. Add audit logs and red-team tests before write access goes live, and give every control a named owner.
Why AI agent security changes once agents can write
A chatbot that answers questions can embarrass you. An agent that can send email, update a CRM record, merge code, issue a refund or call an internal API can hurt you, because the model's output is now an action. The risk moves from "wrong answer" to "wrong action, taken with your credentials".
Two OWASP lists make this split explicit. The OWASP Top 10 for LLM Applications 2025 puts Prompt Injection at LLM01 and Excessive Agency at LLM06. In December 2025 the OWASP GenAI Security Project published the Top 10 for Agentic Applications for 2026, which opens with Agent Goal Hijack (ASI01), Tool Misuse and Exploitation (ASI02) and Identity and Privilege Abuse (ASI03). It adds a useful idea, least agency: unneeded autonomy widens the attack surface without adding value.
For governance language, the NIST AI Risk Management Framework (AI RMF 1.0, released January 26, 2023, with a Generative AI Profile added July 26, 2024) organizes the work into Govern, Map, Measure and Manage. This post stays in Measure and Manage: the controls engineers build. With Q1 roadmaps being set now, this is the right moment to fund them before agents get write scopes in January.
Next question: How do senior AI pod teams keep production agents safe and reliable?
Prompt injection: assume it gets through
OWASP's LLM01 entry separates two forms. Direct injection comes from the person typing to the agent. Indirect injection arrives inside content the agent reads: a web page, a PDF, a support ticket, a calendar invite, a code comment or an email. Indirect injection is the one that matters for agents, because the attacker never needs an account.
The agentic Top 10 describes a real case it calls EchoLeak: a crafted email that caused Microsoft 365 Copilot to follow hidden instructions and leak confidential emails, files and chat logs with no user interaction.
OWASP is direct that retrieval and fine-tuning are not a fix; it says research shows they "do not fully mitigate" prompt injection. So build in layers:
- Separate instructions from data. Mark retrieved content as untrusted in the prompt structure, and never let it change the agent's goal, tool list or recipients.
- Constrain outputs. Require structured, schema-validated tool calls instead of free text that gets parsed into actions.
- Split reading from acting. A model that reads untrusted documents should not be the same component that holds write tools. A 2025 paper by a group of security researchers, Design Patterns for Securing LLM Agents against Prompt Injections, makes this case: constrain the architecture so injected text cannot trigger consequential actions, and accept some loss of flexibility for it.
- Filter, but do not rely on filters. Input and output classifiers catch known patterns. They are a speed bump, not a wall.
Next question: How do you test an AI agent against prompt injection before launch?
Tool permissions: design for least agency
Most agent incidents are permission problems wearing a prompt costume. OWASP's Excessive Agency entry names three root causes: excessive functionality (tools the task does not need), excessive permissions (tools with broader downstream rights than needed) and excessive autonomy (high-impact actions with no independent check).
Controls that hold up in production:
- One tool per job. Replace "run shell command" or "call any API" with narrow functions such as create_draft_reply or update_ticket_status. Open-ended tools turn every injection into remote execution.
- Read-only by default. Grant write scopes per tool, per environment, and only where a use case needs them.
- Act as the user, not as a service account. The agent should inherit the requesting user's permissions, never more.
- Enforce authorization outside the model. OWASP calls this complete mediation: the downstream system checks permission on every call, so a convinced model still cannot do what the user could not.
- Deny unknown tools. The OWASP AI Agent Security Cheat Sheet says to fail closed for unknown tools.
Next question: What does it cost to hire a senior AI agent developer?
Data exfiltration and credential handling
Exfiltration is the usual goal of an injection: get sensitive data somewhere the attacker can read it. Common channels are an outbound email, a web request with data in the URL, a rendered image link, a public ticket comment or a file share.
- Minimize context. Only load the records the task needs. Classify data and redact or mask by class before it reaches the model.
- Control egress. Allowlist outbound domains and recipients for any tool that can send data out. Block arbitrary URL fetches from agents that also read sensitive data.
- Screen outputs. Check tool arguments and final answers for secrets, personal data and exfiltration patterns before execution, as the cheat sheet's output validation section advises.
- Keep secrets out of the model. Credentials live in a vault and are injected by the tool runtime, never pasted into prompts, memory, logs or test fixtures.
- Short-lived, task-scoped tokens. Issue credentials per task with tight scopes and expiry. Orchestrators should never pass their full credentials to sub-agents, which is the pattern ASI03 warns about.
- Keep a kill switch. One action should revoke an agent's credentials and pause its queue.
Next question: What does a senior security engineer own on an AI agent project?
Human approval on irreversible actions
Approval is the control that turns "the agent did it" into "a person approved it". It works only if the approval is specific. The OWASP cheat sheet recommends binding each approval to the exact actor, tool, target and parameters, with an expiry, and failing closed if validation or logging fails.
A practical tiering most teams can adopt:
- Tier 0, read: no approval, logged.
- Tier 1, reversible write (draft, tag, internal note): no approval, logged, sampled for review.
- Tier 2, external or costly (send to a customer, change a price, open a production change): human approval of the exact payload.
- Tier 3, irreversible (delete, pay, transfer, grant access): approval plus step-up authentication, and often two people.
Watch for approval fatigue. If reviewers click "approve" on hundreds of items a day, the control is theater. Show the diff, the source of each instruction and what will change, and keep tier 2 volume low enough to read.
Next question: Who builds approval flows inside our own cloud and repos?
Audit logs and observability
When something goes wrong, you need to answer three questions fast: what did the agent see, what did it decide and what did it do. Log every model call, retrieved document ID, tool call with arguments, approval ID and outcome, with sensitive values redacted. Record the prompt, policy and model versions so you can reproduce a run.
Then use the logs. Alert on unusual tool-call rates, new outbound destinations, repeated denied calls and cost spikes. OWASP notes that logging and rate limits do not prevent excessive agency, but they limit the damage.
Next question: What does LLM evaluation and observability work include?
Red-teaming and evaluation before launch
OWASP lists adversarial testing as a core prompt injection mitigation, and AI red teaming is how you find out whether your layers work together. Build a pre-launch agent test suite, run it in CI and rerun it whenever the model, prompt, tools or credential scopes change.
- Plant hidden instructions in documents, emails, tickets and web pages the agent will read.
- Try to get the agent to call a tool outside its task, or a permitted tool with hostile arguments.
- Attempt exfiltration through every outbound channel the agent has.
- Try to bypass approval by splitting one risky action into several small ones.
- Poison memory in one session and check whether it affects another user.
Track pass rates as a release gate, the same way you treat failing unit tests.
Next question: What do LLM evaluation engineers test for in agents and guardrails?
The control map: risk, control and owner
Every control needs a named owner, or it decays the first time a deadline hits. This is the map to agree on before any agent gets write access.
Pre-write-access checklist:
- Every tool listed, with its scope and the user identity it runs as.
- Every irreversible action mapped to an approval tier.
- Egress allowlist in place for any agent that reads sensitive data.
- Credentials vaulted, scoped and revocable in one step.
- Audit logs flowing and an on-call owner for agent alerts.
- Red-team suite passing in CI.
Next question: How much does a senior AI engineer cost per month?
How Ryz can help
Ryz AI pod teams are senior engineers who build agents inside your cloud and repos and can design the controls above in from the first sprint, and you can add senior security engineers to own exfiltration, identity and red-team work. Pods have shipped an AI voice platform with 1M+ outbound calls and an AI marketing-compliance system covering 8,000+ documents. Only the top 1% make it, they work US business hours, and you own all code and IP.
FAQ
What is the biggest AI agent security risk?
Indirect prompt injection combined with broad permissions. Injection alone is a nuisance; injection plus write tools, shared credentials and open egress is a breach. Reduce the permissions first, because you cannot fully filter the injection.
Is the OWASP Top 10 for LLM Applications enough for agents?
It is the right starting point, especially LLM01 and LLM06. For agents that plan and act across tools, also use the OWASP Top 10 for Agentic Applications for 2026 and the AI Agent Security Cheat Sheet, which cover goal hijack, identity abuse, memory poisoning and inter-agent trust.
Do we need human approval for every agent action?
No. Approve by tier: reads and reversible writes run on their own with logging, while external, costly or irreversible actions need a person to approve the exact payload. Approving everything creates fatigue and weakens the control.
Who should own AI agent security?
Shared, with named owners per control. AI engineering owns prompt structure and tool design, security engineering owns exfiltration and red-teaming, the platform team owns credentials, and the workflow's product owner owns approval rules.
