In plain words
Software tests often feed programs into compilers and libraries to find bugs, but generated programs may be rejected before they exercise deeper behavior.
Typed Template Fuzzing asks a language model to mark types and available variables in example programs, then Typos changes marked parts while restoring the original text around them.
Across many tested languages, Typos produced mostly accepted inputs. In a six-month campaign, the authors reported 135 bugs, 128 previously undiscovered. These results describe the tested systems, not every program or language.
The research
For compilers and libraries with complicated input formats, can a general fuzzer generate programs the software accepts without requiring experts to build language-specific parsers and type checkers? Rejected inputs cannot exercise the deeper program behavior that fuzzing seeks to test.
Typed Template Fuzzing turns seed programs into repaired, typed templates that Typos mutates into software-accepted inputs without language-specific parsers.
Typos first asks an LLM to annotate seed programs, then repairs surrounding text from each original seed and mutates compatible parts. In a six-month campaign across 27 targets in 21 languages, the authors reported 135 bugs, including 128 previously undiscovered.
Over a seven-day window, Typos’s validity rate averaged 96% across the 27 systems. Validity is the share of unique mutants accepted with a zero exit code. Together, the campaign reports show that Typos generated many usable inputs and found bugs in the selected systems.
These results cover the selected targets and experiments. The authors note that target selection limits how far the findings generalize, and errors inside LLM-marked terms can still affect templates.
Analysis
Deep
- Original paper authors
- Lu Maltsis, Dominik Winterer
- Submitted to arXiv
- License
- CC BY 4.0