Such a system may plan steps, retain memory, select tools and act for a user. Its security boundary includes the model, orchestration code, prompts, memory, tool interfaces, credentials, connected data, other agents and the action environment.
Risks include indirect prompt injection, manipulated tool results, excessive permissions, credential exposure, poisoned memory, cross-agent impersonation and harmful chains of permitted actions. Failures can also occur without an attacker when an agent misunderstands a goal or optimizes the wrong objective. Controls should limit what the agent can observe and change, with stronger approval for consequential actions.
Key points
Bound authorityGive each tool and workflow narrowly scoped, short-lived credentials; restrict destinations, data, action types, spending and execution time; and require approval for high-impact steps.
Context and memory protectionSeparate users and tasks, label and validate untrusted content, control what persists, record provenance and prevent retrieved instructions from silently overriding authorized goals.
Observation and containmentLog prompts, decisions, tool calls and results where proportionate; detect abnormal action chains; use sandboxes, rate limits, transaction checks and reliable stop or rollback mechanisms.
Whole-system assuranceTest realistic multi-step attacks and accidental failures, including tool compromise, memory poisoning, cascading agents and recovery from partially completed actions.
Important limitationAgentic AI security is an emerging field with unsettled terminology, protocols and assurance methods. Conventional software, identity and AI risk controls remain necessary, but they may not fully address delegated and adaptive behavior.