A speech recognizer learns to keep committed words unchanged
Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription · arXiv:2611.06043
In plain words
A streaming speech recognizer sends words out while someone is still talking. Those words can change when more sound arrives, making other software redo work. R2T2 is trained to keep text it has committed and to wait when the next words are uncertain.
In tests using 160 ms audio chunks, R2T2 made fewer recognition errors than its base model across 13 English and Chinese test sets. That result covers those tests and settings; it does not measure real serving time.
The research
A changing live transcript can make connected software redo work or act on words that will later disappear. A recognizer that commits only stable text addresses that specific problem; the reported accuracy comparison shows the approach can also reduce errors in the tested streaming settings.
R2T2 adapts a full-audio speech recognizer to commit stable text as audio arrives, using labels that teach it when to wait on uncertain words.
Researchers compared R2T2 with its base model on the same 13 English and Chinese test sets, using 160 ms chunks and one provisional token. The equal-weight macro error was 5.76% for R2T2 and 21.01% for the base model.
Because R2T2 was lower on every listed set, the comparison supports lower recognition error for the post-trained system in this streaming setup. The average combines English word and Chinese character error percentages, weighting each dataset equally.
Emission lag was measured on the audio timeline assuming recognition kept pace; runtime serving time was not measured. The tests cover the reported English and Chinese sets and 18 additional-language sets, not deployment performance or broader generalization.
What the evidence shows
- Stability is the goal
Confucius4-R2T2 (R2T2) is a streaming speech recognition model trained to release text in stable prefixes: once text is committed, later updates do not change it. That addresses a specific cost of streaming recognition, where revised words can force downstream software to redo work or act on a transcript that later changes. Stability governs whether output is revised; accuracy remains a separate question.
- A rule for committing text
R2T2 uses Longest Stable Prefix learning to decide which partial text to commit. A label is the longest earlier output that matches the beginning of the model’s own transcript for the full utterance; a Wait token marks an uncertain continuation.
The paper proves that these labels never shrink and are the longest that satisfy both conditions. The proof concerns commitment behavior, not whether the transcript is correct.
- A responsiveness diagnostic
On a selected set of English utterances, Qwen3-ASR-1.7B had the highest normalized streamability score (0.493) among the four models.
The score tracks how much each model’s partial transcript matches its own complete-audio transcript as audio arrives, so the comparison indicates relative responsiveness, not word accuracy.
- Lower error in the main tests
At matched chunk and provisional-token settings, R2T2’s equal-weight macro recognition error across the same English and Chinese test sets was lower than its base model’s (5.76% versus 21.01%).
This supports lower recognition error for the post-trained system in these tested streaming conditions, though the comparison does not identify which post-training component accounts for the difference.
- Transfer with bounded evidence
On 18 test sets in ten languages absent from R2T2’s post-training, its recognition error was lower than the base model’s on every set at the same chunk and token settings. The timing measure, however, records when matching transcript text becomes stable on the audio timeline, assuming inference keeps pace.
It does not include runtime serving time, and deployment performance and broader generalization remain to be evaluated.
Deep
- Original paper authors
- Daniel Huang, Qi He, Shaoxiong Duan, Yingjie Li, Qiao Cheng
- Submitted to arXiv
- License
- arXiv non-exclusive