Checking each action can stop agents’ unauthorized acts
In plain words
The paper introduces CAPE-Bench, a set of paired tasks where one fact changes whether an agent may take an action, plus a way to check an agent before it calls a tool. The check reads the agent’s internal state and can steer a flagged action; repeat flags can prompt an approval request.
Across four open-weight model families, checking every action reduced violations over the episode. In matched tasks where the action was allowed, asking for approval on repeat flags kept completion close to runs without the gate. The tools were simulated.
The research
Can oversight stop an agent that has already seen an action is unauthorized, without merely delaying the violation or blocking the same work when it is allowed? A single-turn check cannot answer both; the answer depends on following the episode in paired worlds.
The authors introduce CAPE-Bench and an internal-state gate that forecasts unauthorized actions, checks each proposed tool call, and asks for approval on repeated flags.
CAPE-Bench pairs mock-tool worlds differing in one fact that decides whether a critical action is allowed. An internal-state forecaster flags proposed calls so a gate can steer them or seek approval on repeated flags.
Across four open-weight model families, checking every call reduced eventual violations; gate-then-ask kept authorized completion close to paired runs without a gate.
Because researchers followed episodes and checked environment outcomes, the results address delay and lost allowed work in these tests.
The evaluations used mock tools and four open-weight model families; the gate threshold was set empirically and has no distribution-free false-alarm guarantee. The transfer test covered two model families, and the paper did not test agents trained to evade the readout or a live model-approver loop.
Analysis
Deep
- Original paper authors
- Arya Fayyazi, Mehdi Kamal, Massoud Pedram
- Submitted to arXiv
- License
- CC BY 4.0