INFUSE tests how hidden instructions redirect agents
Questa pagina non ha ancora una traduzione verificata. È mostrata la versione inglese. Leggi la versione inglese
In parole semplici
An agent may read a file or terminal message while doing a task. If that content contains hidden instructions, it may follow them instead of the user’s request. INFUSE is a test suite that places such instructions in task environments and records what the agent encounters and does.
Across 19 models using Mini-SWE-Agent, a language-model judge marked a mean 19.6% of undefended attack runs compromised, requiring the instruction to affect actions or decisions. Separate automatic checks looked for concrete results such as stolen files or changed credentials.
La ricerca
When an agent works through a multi-step task, it may need to read files and tool output it does not control. If an attacker hides instructions there, can they steer the agent, and can an evaluation distinguish merely seeing those words from acting on them?
INFUSE is an end-to-end test suite that places hidden instructions in task environments and tracks how they affect agents’ actions.
INFUSE pairs 40 tasks with attack scenarios, places instructions in compatible environment content, and records what agents read, write, and execute. A language-model judge and automatic checks for concrete attack artifacts score each run.
Across 19 models in Mini-SWE-Agent, mean undefended-run compromise rates were 19.6% by judge and 15.4% by artifact checks. Both measures show compromise in the tested workflows, though they capture different signals.
The defense result came from a high-vulnerability subset with incomplete coverage, while cross-harness replays included only cells already compromised in Mini-SWE-Agent. The English-only, hand-authored scenarios and sandbox do not establish rates for other agents or production settings.
Analisi
Dettagli
- Autori del paper originale
- Thuy Dung Nguyen, Jingyi Guo, Rana Seda Dubauskas, Sreepriya Damuluru, Taylor T. Johnson
- Inviato ad arXiv
- Licenza
- CC BY 4.0