Agent approval can fail when the displayed action is not bound
Questa pagina non ha ancora una traduzione verificata. È mostrata la versione inglese. Leggi la versione inglese
In parole semplici
When an AI agent asks permission to use a tool, the approval can be misleading if the agent controls the description or changes the call after approval. The authors built Agent Step-Up, a gateway that makes the approval card from the tool call and runs that same stored call after approval.
In task replays, a simulated approver comparing the card with the task caught more forged actions with the gateway’s card than with agent-written descriptions. The study did not test human approvers.
La ricerca
When an agent asks for approval, what must that approval establish: that someone clicked, or that the exact action later executed was the one they saw? The agent may control its description, or the action may change after approval.
The authors define effect-bound approval and build Agent Step-Up, a gateway that renders one tool call, binds approval to it, and executes that stored call.
The authors replayed task scenarios in which an agent described a pending user action but requested an injected one. They counted an attack when its action appeared in a mock resource ledger.
Across four task suites, a simulated approver comparing the card with the task saw success in 0.54 of pairs with agent-written cards and 0.03 with ASU.
The remaining ASU successes were actions allowed without approval. This result supports binding the displayed action to the call that executes, when the approver checks the card against the task.
The attack replays used simulated approvers, and the study included no human subjects, so the results do not show how people would respond. The formal guarantees also depend on the approver using trusted card fields, closed first-credential enrollment and an honest card-rendering page.
Analisi
Dettagli
- Autori del paper originale
- Sai Varun Kodathala, Prashanth Pollishetty, Jaylen Cargill
- Inviato ad arXiv
- Licenza
- CC BY 4.0