AI Security & Safety
Research on AI Security & Safety: explanations, evidence, limitations and connections between papers.
37 results
Match each safety claim to the evidence that survives the handoff
A repainted image may still resemble its source after an invisible ownership mark weakens. A coding agent may likewise carry a plausible summary or file forward after losing the origin of a request. The result alone leaves a gap in what a later check can establish. Across the studied settings, evidence has to travel with the claim it supports. Speaker and artifact history matter for later actions; component records matter for runtime identity; build inputs matter for exact reproduction. Each study tests a different link in that chain.
Certifying robustness when language model routes change
CertMoE accounts for changing expert routes and reduced attack-found certificate violations in a scoped text-classification evaluation.
How harmful records can evade data selectors
A study shows how suppliers can shape harmful records to pass data selection, and evaluates a rank-based defense in bounded tests.
Forged labels can turn tool text into user instructions
A study finds forged message labels can corrupt coding-agent memory, while a code check catches some false approvals.
Why some purpose violations require comparing agent runs
A formal analysis shows why one-action guards miss relational purpose violations and how paired monitoring reveals them.
AutoRedClaw tests attacks on persistent agent state
AutoRedClaw generates checked attack tests; its frozen collection reveals persistent-state risks in selected agent systems.
A small RAG detector uses context to spot hidden attacks
A small local detector checks queries against retrieved text and performs well on tested instruction attacks, but not factual poisoning.
Post-quantum TLS can be faster on small devices
A wolfSSL study finds ML-KEM-512 handshakes faster than X25519 across four devices when authentication stays matched.
A kernel gate ties agent approvals to each connection
Connection Escrow holds agent connections for socket-bound approval; one cluster test blocked a raw-socket bypass.
REX moves hazardous knowledge into removable experts
REX moves hazardous knowledge into removable experts; after training, it scored lowest among usable methods in six tested settings.
Match an automated pass to the failure paths it tested
A faster training recipe, a warning from a code checker, and a patch that blocks an exploit can each receive an encouraging automated result. The practical tension is whether that result covers the failure that matters for the next decision. Across these studies, a pass becomes useful when its test case and target are clear. Their checks screen settings, filter generated programs, or replay attacks, yet each leaves paths outside what it exercised. Read the pass as evidence for those tested paths.
Intact ties browser actions to the webpage elements behind them
Intact ties browser agent actions to page elements, reducing injected attack steps in tests on real webpages.
LLM4SAST finds faults in six Java and Rust code analyzers
LLM4SAST tests Java and Rust security scanners with generated code changes, reporting 66 candidate faults across six tools.
Translated requests can change agent guards’ safety verdicts
LinguaGuardBench finds translated requests shift many agent guards’ verdicts, with results bounded to tested steps and guards.
Agent approval can fail when the displayed action is not bound
A study tests agent approval attacks and finds trusted action cards cut forged-action success in simulated task replays.
INFUSE tests how hidden instructions redirect agents
INFUSE tests hidden instructions in agents; judge labels and artifact checks both find compromise in undefended task runs.
A framework tests local tools for filesystem leaks
MCP-FSD tests local tool servers for out-of-scope file disclosure and whether inferred sandbox rules contain it.
Output scores can miss changes to an AI agent’s runtime stack
DIB-Bench tests whether agent output scores reveal runtime changes; manifest hashes detect declared component changes in its synthetic study.
Profiling reveals capabilities that threat labels miss in agent skills
A two-phase method found elevated capabilities in seven in ten listings sampled from two agent skill marketplaces.
JevProbe finds frequent decision flips in tested Jev apps
JevProbe traces attacker-controlled context into application decisions and finds frequent flips in tested open-source Jev apps.
P2Patch finds bypasses and strengthens vulnerability repairs
Compared with single-pass repair, P2Patch improves coverage on independent tests by adding confirmed bypasses as requirements.
Signed model weights do not make quantized files reproducible
A study finds missing build details block exact checks of quantized models and tests statistical detection limits.
A publisher’s identity can help agents screen claims about rivals (F1)
A standing check screens rival claims in catalogue text; tests found sentence removal stopped most attacks that changed agent choices.
Kairos holds uncertain agent actions for later review
Kairos holds uncertain agent actions for later review, reducing successful attacks in AgentDojo while preserving benign task completion.