Security bugs can hide inside valid LLM tool calls

When a language model passes a tool call through an orchestrator, what can the tool safely assume about the arguments? A call may obey the schema while an attacker controls its contents, and the tool may return normally after violating its documented security meaning. The paper asks whether testing the tool’s semantics can expose this hidden risk.

ToFu turns tool documentation into executable semantic checks and schema-valid fuzzing tests, then uses observed effects and outputs to find silent security violations.

The framework maps each argument’s documented meaning to registered security checks, mutates arguments while preserving the tool’s schema, and judges runtime effects or whether a flagged input was safely rejected.

This tests the contract at the implementation boundary, rather than asking only whether the process crashes.

Across 273 reachable servers, ToFu produced 508 candidates; manual review and deduplication left 253 verified bugs affecting 343 tools in 118 servers.

Because 182 candidates involved successful execution despite a violated semantic constraint, the findings support the paper’s conclusion that valid formatting and normal completion do not establish safe handling; tool metadata needs enforcement at the implementation boundary.

ToFu needs observable runtime evidence and executable tools. It can miss properties with no observable event, servers that fail before their logic runs, and multi-agent, third-party-component, or stateful call-sequence behavior.