Giving AI agents a purpose profile sharpens security decisions
Intent-Aware Protection for AI Agents · arXiv:2611.16122
In plain words
An agent can have permission to use a tool and still be doing something outside its assigned job. This paper introduces Intent-Aware Security, a framework that gives each agent a profile of its purpose, allowed tasks, tools, data limits, and prohibited actions.
The profile guides checks before and after an agent acts, as well as ways to reshape its inputs or instructions. In a synthetic test, a profile-based classifier better distinguished four kinds of interaction than one using raw agent context.
Separate tests found stronger protection in simulated responses and better monitoring on a limited set of executed agent tasks.
The research
A request can be within an agent’s technical permissions but still outside the job it was assigned to do. That gap matters when judging whether an interaction is appropriate for a particular agent.
These results support adding the agent’s intended purpose as context for security checks, while leaving the quality of that context dependent on the source instructions and their extraction.
Intent-Aware Security represents an agent’s trusted purpose in a profile and uses it to guide security checks and protections across the agent’s workflow.
On the synthetic Intent-Aware test set, classifiers judged interactions against the receiving agent’s purpose. The profile-based classifier reached 79% macro-F1, compared with 45% for a raw-context classifier, showing that the structured profile helped separate four intent categories in this test.
Other evaluations extended the comparison to executed trajectories and simulated protections. They also favored profile-based methods in the reported measures, though simulated hardening had lower success on benign requests.
The executed-trajectory test covered four agents using GPT-4o, treated user requests as trusted, and included indirect prompt injections rather than out-of-scope requests or deliberate repurposing.
The framework also assumes its organizational, role, and developer sources are authentic and uncompromised; it does not guarantee they are complete, consistent, or correctly extracted into a profile.
What the evidence shows
- Purpose adds security context
Intent-Aware Security is a framework that represents an agent’s trusted organizational, role, and developer intent in an Intent Profile. It uses that profile in checks before and after execution, input alignment, and system-prompt hardening.
The central question is whether the agent’s purpose helps distinguish legitimate use from requests that exceed its role, repurpose it, or attack it.
- Structured context improves classification
Both classifiers used the same model and interactions, so the comparison tests whether the form of agent-specific context matters.
The profile-based classifier’s stronger four-category result suggests that explicitly representing an agent’s purpose helped distinguish legitimate use, scope violations, repurposing, and attacks in this synthetic test.
- Executed tasks add another test
On executed AgentDojo trajectories, the profile-based monitor better separated successful from failed injected attacks across score thresholds than a raw-configuration monitor.
This supports post-action monitoring in that test, which treated user requests as trusted and covered indirect prompt injections.
- Simulated protections show trade-offs
In simulated responses, profile-based input alignment performed better overall than DataFilter, while profile-based hardening showed only a small overall gain and lower success on benign requests.
Success meant fully refusing out-of-scope, abusive, and malicious requests, and fully completing benign ones, so the hardening result shows a trade-off in this evaluator-scored test.
- The evidence has clear boundaries
The evidence comes from different settings: synthetic, LLM-validated interactions; simulated agent responses judged by an LLM; and a narrower external trajectory test. AgentDojo covered four agents using one model, treated user requests as trusted, and did not include out-of-scope or abusive interactions.
The framework also relies on authentic trusted intent sources, without guaranteeing that they are complete, consistent, or correctly represented in the profile.
Deep
- Original paper authors
- Fady Copty, Neta Haiby, Max Braitmaiere, Julia Korsunsky
- Submitted to arXiv
- License
- arXiv non-exclusive