AI Security & Safety
Research on AI Security & Safety: explanations, evidence, limitations and connections between papers.
33 results
Why some purpose violations require comparing agent runs
A formal analysis shows why one-action guards miss relational purpose violations and how paired monitoring reveals them.
AutoRedClaw tests attacks on persistent agent state
AutoRedClaw generates checked attack tests; its frozen collection reveals persistent-state risks in selected agent systems.
A small RAG detector uses context to spot hidden attacks
A small local detector checks queries against retrieved text and performs well on tested instruction attacks, but not factual poisoning.
Post-quantum TLS can be faster on small devices
A wolfSSL study finds ML-KEM-512 handshakes faster than X25519 across four devices when authentication stays matched.
A kernel gate ties agent approvals to each connection
Connection Escrow holds agent connections for socket-bound approval; one cluster test blocked a raw-socket bypass.
REX moves hazardous knowledge into removable experts
REX moves hazardous knowledge into removable experts; after training, it scored lowest among usable methods in six tested settings.
Match an automated pass to the failure paths it tested
A faster training recipe, a warning from a code checker, and a patch that blocks an exploit can each receive an encouraging automated result. The practical tension is whether that result covers the failure that matters for the next decision. Across these studies, a pass becomes useful when its test case and target are clear. Their checks screen settings, filter generated programs, or replay attacks, yet each leaves paths outside what it exercised. Read the pass as evidence for those tested paths.
Intact ties browser actions to the webpage elements behind them
Intact ties browser agent actions to page elements, reducing injected attack steps in tests on real webpages.
LLM4SAST finds faults in six Java and Rust code analyzers
LLM4SAST tests Java and Rust security scanners with generated code changes, reporting 66 candidate faults across six tools.
Translated requests can change agent guards’ safety verdicts
LinguaGuardBench finds translated requests shift many agent guards’ verdicts, with results bounded to tested steps and guards.
Agent approval can fail when the displayed action is not bound
A study tests agent approval attacks and finds trusted action cards cut forged-action success in simulated task replays.
INFUSE tests how hidden instructions redirect agents
INFUSE tests hidden instructions in agents; judge labels and artifact checks both find compromise in undefended task runs.
A framework tests local tools for filesystem leaks
MCP-FSD tests local tool servers for out-of-scope file disclosure and whether inferred sandbox rules contain it.
Output scores can miss changes to an AI agent’s runtime stack
DIB-Bench tests whether agent output scores reveal runtime changes; manifest hashes detect declared component changes in its synthetic study.
Profiling reveals capabilities that threat labels miss in agent skills
A two-phase method found elevated capabilities in seven in ten listings sampled from two agent skill marketplaces.
JevProbe finds frequent decision flips in tested Jev apps
JevProbe traces attacker-controlled context into application decisions and finds frequent flips in tested open-source Jev apps.
P2Patch finds bypasses and strengthens vulnerability repairs
Compared with single-pass repair, P2Patch improves coverage on independent tests by adding confirmed bypasses as requirements.
Signed model weights do not make quantized files reproducible
A study finds missing build details block exact checks of quantized models and tests statistical detection limits.
A publisher’s identity can help agents screen claims about rivals (F1)
A standing check screens rival claims in catalogue text; tests found sentence removal stopped most attacks that changed agent choices.
Kairos holds uncertain agent actions for later review
Kairos holds uncertain agent actions for later review, reducing successful attacks in AgentDojo while preserving benign task completion.
Jev’s resistance to attacks changes across tasks
A shared adversarial test finds Jev resists forged inputs on moderation but is more vulnerable on phishing.
AegisAsync traces Python async behavior to flag risks
A bytecode analysis links Python async events to risks, finding reviewed bugs in a stratified package sample.
Small image changes can steer preference training
DPOison-VL shows small training-image changes can redirect preference alignment in tested vision-language models.
Match agent security checks to the evidence they can observe
An assistant can finish a professional task while also taking an unauthorized action. A tool call can appear approved while its server accesses a file beyond the tool’s intended scope. The visible deliverable and the approval prompt leave part of the story unseen. The first clue is the route by which influence enters. A workspace instruction must be encountered, a tool argument can select a file, coordinated queries must stand out from ordinary traffic, and a saved artifact can shape a later session. Each route leaves different evidence.