Rondo tests compiler bugs that depend on earlier calls
Questa pagina non ha ancora una traduzione verificata. È mostrata la versione inglese. Leggi la versione inglese
In parole semplici
Deep learning software can compile a model once and reuse the compiled code on later calls. That reuse can go wrong after inputs or model state change. Rondo is a testing method that runs ordered calls, then checks whether a later answer changes when the compiler’s remembered information is cleared.
The authors tested it on four frameworks and 24 model architectures. It found 117 bugs, 35 of them dependent on earlier calls; developers confirmed 16 of the 18 reports about those stateful bugs. These results show the method found such bugs in the tested systems, not how many remain.
La ricerca
Can a compiler test distinguish an error caused by information retained from earlier model calls from one caused by the current inputs or model state? Single-call tests do not represent that history, and comparing compiled with uncompiled results alone cannot identify the cause.
Rondo searches ordered model-call sequences and checks the same later call with compiler state kept or reset to identify state-dependent errors.
To separate compiler memory from changes in the model itself, Rondo replays the same later call with its inputs and relevant program state fixed. It compares compiled runs with remembered compiler state retained and reset, alongside an uncompiled run.
The authors first analyzed 26 publicly reported cases; all were reproduced on affected versions, and each involved earlier calls followed by a later incorrect result.
Across four frameworks and 24 architectures, Rondo detected 117 bugs, including 35 stateful ones. Developers confirmed 35 of 39 submitted reports, including 16 of 18 stateful reports. This shows the method found bugs developers accepted in those systems, though not how many bugs exist overall.
The evaluation covers selected programs, four frameworks, and the reported hardware and settings. Its bug counts and coverage results do not establish how many bugs exist overall or how Rondo performs on other programs and systems.
Analisi
Dettagli
- Autori del paper originale
- Jiatao Ge, Ruixiang Qian, Yibo Hu, Yanzhou Mu, Yinglong Zou
- Inviato ad arXiv
- Licenza
- arXiv non-exclusive