False safety claims can mislead code-auditing models
Trusting the Tool: Asymmetric Deference of LLM Vulnerability Auditors to Static-Analysis Evidence · arXiv:2610.32876
In plain words
When language models check C/C++ code for vulnerabilities, they may receive notes from a program analyzer. This paper presents a controlled method for testing whether models check those notes against the code, especially when a note says a safety check exists or is missing.
For one tested model, false notes claiming a check existed made it miss more vulnerable functions; false alarms about missing checks did not lower accuracy. A separate verification step helped that model in a later test.
The research
When a language model checks code with help from an analyzer, does it verify the analyzer’s claims against the code? The answer matters because a note saying a safety check exists may invite a different response from one saying the check is missing. The study tests whether that difference appears when the claims concern the same code operations.
The authors compare language models’ audits with true and fabricated analyzer notes to test whether models scrutinize safety claims differently from danger claims.
The authors compared audits of the same vulnerable or patched C/C++ functions under different analyzer notes. For deepseek-chat, false claims that a check existed lowered accuracy on vulnerable functions from 0.75 with true relevant notes to 0.64; false missing-check alarms raised accuracy on patched functions from 0.67 to 0.79. These paired comparisons changed the truth of notes about the same operations.
Because models may differ in their baseline tendency to flag code, the authors also measured shifts toward each false claim relative to true notes. In a replication with seven new models, the average shift favored false assurances. This supports an asymmetry in how the tested models responded to the two kinds of claims.
These findings come from isolated audits of short C/C++ functions using evidence constructed by the authors’ analyzer. The proposed verification step improved results in a pre-specified test for deepseek-chat, but its smaller gain for qwen-plus was not statistically significant.
What the evidence shows
- The question about evidence
The paper presents a controlled method for studying how language models use static-analysis evidence when checking C/C++ functions for vulnerabilities. Its question is whether models compare claims about code checks with the code itself, and whether they scrutinize claims that a check exists differently from claims that one is missing. This isolates how models respond to evidence, rather than measuring vulnerability detection alone.
- Building checkable notes
To make the truth of each note known, the authors built an analyzer that lists sensitive operations and the conditions that hold on every path to them, along with concerns the function does not rule out. They paired vulnerable and patched functions, then varied the notes shown to models.
False claims could be checked against the function; the model did not see the patch. This made it possible to compare false notes with true notes about the same operations.
- Truth changed the effect
True analyzer notes produced little measurable change in accuracy in the reported DeepSeek flash configurations: showing all true notes changed accuracy by +0.02 for each, not statistically significant. The sharper comparison held the function and note relevance fixed while changing whether a note was true.
For deepseek-chat, false claims that a check existed lowered accuracy on vulnerable functions, while false missing-check alarms did not lower accuracy on patched functions.
- Comparing verdict shifts
Accuracy alone can reflect a model’s existing tendency to label code vulnerable. The authors also measured whether false notes shifted verdicts toward their own claims, relative to true notes about the same operations.
Across seven new models in an internally registered replication, the mean asymmetry was positive: false assurances moved verdicts toward “safe” more than false alarms moved them toward “vulnerable. ” The size varied across models.
- Testing a verification step
The authors interpret the results as a difference in scrutiny: a claim that a check is missing can prompt a search for it, while a claim that it exists can be accepted without checking. In a pre-specified test, a separate call that verified notes and showed refuted claims improved deepseek-chat’s accuracy under false assurances without changing its accuracy under true relevant notes. The gain for qwen-plus was smaller and not statistically significant.
Deep
- Original paper authors
- ZhiHong Liu, Shuyao Gao
- Submitted to arXiv
- License
- arXiv non-exclusive