A new test separates seeing an attack from carrying it out

When an agent works among shared files and messages, an attacker may control only some resources, and the agent may never open them. If it does encounter a malicious instruction, does it ignore it, or treat it as authorized? Measuring only whether the requested work was completed cannot answer that safety question.

APEX-Harm tests how workspace instructions reach long-task agents and measures both exposure to attacks and unauthorized actions across six attack types.

The authors built 180 attack cases from 15 professional tasks, varying how malicious instructions or scripts reach an agent. Evaluators separately measured whether tool responses contained injected instructions, whether unauthorized actions succeeded, and whether agents met task rubrics.

Across 12 models, task-specific helper replacements had average attack success of 84% for direct-action attacks and 81% when runtime information was needed. Opus 5 encountered static injected instructions in 97% of runs, but attacks succeeded in 7%, showing exposure and action are distinct.

The findings cover 15 tasks in investment banking, law, and consulting, selected attack types, and synthetic authorization scenarios. Each model and attack case was run once, so the results do not show run-to-run variability or establish how agents behave in other domains.