AI Security & Safety
Research on AI Security & Safety: explanations, evidence, limitations and connections between papers.
15 results
A publisher’s identity can help agents screen claims about rivals (F1)
A standing check screens rival claims in catalogue text; tests found sentence removal stopped most attacks that changed agent choices.
Kairos holds uncertain agent actions for later review
Kairos holds uncertain agent actions for later review, reducing successful attacks in AgentDojo while preserving benign task completion.
Jev’s resistance to attacks changes across tasks
A shared adversarial test finds Jev resists forged inputs on moderation but is more vulnerable on phishing.
AegisAsync traces Python async behavior to flag risks
A bytecode analysis links Python async events to risks, finding reviewed bugs in a stratified package sample.
Small image changes can steer preference training
DPOison-VL shows small training-image changes can redirect preference alignment in tested vision-language models.
Match a security check to the attack path it can observe
An agent can finish a professional task while following a malicious workspace instruction. A campaign spread across accounts leaves a different trace, while a saved artifact can carry an instruction into a later session. A security check needs a signal from the path the attack actually takes. The studies offer evidence at different points: action in a workspace test, coordination in query traffic, and use of an artifact before an operation. Each result answers a narrower question than whether a workflow is secure.
Check what a security score actually observes
A patch can stop a known crash while changing behavior the developer meant to keep. A tool can finish normally after crossing a boundary described in its interface. A later agent can complete its task while carrying harmful content from an earlier artifact forward. The three studies check these failures through different observations. Their findings are most useful when read alongside what the test could see, what counted as failure, and the conditions under which the result was measured.
A new test separates seeing an attack from carrying it out
APEX-Harm tests workspace attacks on professional agents and finds that task-specific script replacements often caused unauthorized actions.
A query-only monitor finds coordinated accounts despite prompt padding
A query-only monitor detects padded campaigns in later WildChat tests, though tested colluder shares were far above production levels.
AgentValve caps Sybil-driven payment exposure by trust capacity
AgentValve uses trust capacity to bound Sybil-driven payment exposure in permissionless AI markets.
NOMOS makes written policies safer for tool-using agents
NOMOS verifies extracted rules and reduces policy violations, but coverage and trusted tool data still limit its guarantees.
Security bugs can hide inside valid LLM tool calls
ToFu turns tool metadata into security tests and finds verified bugs that valid calls can hide.
Persistent artifacts let attacks cross agent sessions
A runtime defense restores artifact provenance and reviews operations, sharply reducing cross-session attacks in tested workflows.
Ordinary web posts can enter AI-search answers
A measurement framework finds ordinary posts can enter AI-search citations under tested conditions, without showing that the posts are accurate.
Why vulnerability patch scores can miss the intended fix
A study finds that crash-test success can miss developer tests, exposing limits in how vulnerability patches are scored.