security safety
Research on security safety: explanations, evidence, limitations and connections between papers.
15 results
A publisher’s identity can help agents screen claims about rivals (F1)
A standing check screens rival claims in catalogue text; tests found sentence removal stopped most attacks that changed agent choices.
Kairos holds uncertain agent actions for later review
Kairos holds uncertain agent actions for later review, reducing successful attacks in AgentDojo while preserving benign task completion.
Jev’s resistance to attacks changes across tasks
A shared adversarial test finds Jev resists forged inputs on moderation but is more vulnerable on phishing.
AegisAsync traces Python async behavior to flag risks
A bytecode analysis links Python async events to risks, finding reviewed bugs in a stratified package sample.
Small image changes can steer preference training
DPOison-VL shows small training-image changes can redirect preference alignment in tested vision-language models.
Where an AI Attack Becomes Visible
An agent may finish a client task after reading a planted file. An attacker may instead spread model-copying queries across accounts, or leave misleading material in a file that survives into a later session. Each route raises a different question about what a security control can see. The studies offer a bounded answer: workspace tests can check whether content reaches an agent and prompts an unauthorized act; query monitoring can detect some coordinated campaigns; review around saved artifacts can help agents reconsider actions. Their results measure different outcomes in different settings.
What Does a Passing AI Safety Check Show?
A passing check can answer a narrow question while leaving the question a reviewer cares about unsettled. A readable account of an audio answer can omit event order; a patch can stop a known crash without meeting a maintainer’s test. Across these studies, the useful first question is what the setup made visible and what counted as success. Their results give evidence about particular audio accounts, patching conditions, tool executions, and attacks carried into later tasks. The scope of each claim follows those conditions and checks.
A new test separates seeing an attack from carrying it out
APEX-Harm tests workspace attacks on professional agents and finds that task-specific script replacements often caused unauthorized actions.
A query-only monitor finds coordinated accounts despite prompt padding
A query-only monitor detects padded campaigns in later WildChat tests, though tested colluder shares were far above production levels.
AgentValve caps Sybil-driven payment exposure by trust capacity
AgentValve uses trust capacity to bound Sybil-driven payment exposure in permissionless AI markets.
NOMOS makes written policies safer for tool-using agents
NOMOS verifies extracted rules and reduces policy violations, but coverage and trusted tool data still limit its guarantees.
Security bugs can hide inside valid LLM tool calls
ToFu turns tool metadata into security tests and finds verified bugs that valid calls can hide.
Persistent artifacts let attacks cross agent sessions
A runtime defense restores artifact provenance and reviews operations, sharply reducing cross-session attacks in tested workflows.
Ordinary web posts can enter AI-search answers
A measurement framework finds ordinary posts can enter AI-search citations under tested conditions, without showing that the posts are accurate.
Why vulnerability patch scores can miss the intended fix
A study finds that crash-test success can miss developer tests, exposing limits in how vulnerability patches are scored.