Earlybird predicts words before speech ends
Earlybird: Predictive Streaming Speech Recognition in 29M Parameters · arXiv:2611.14421
In plain words
Most streaming speech recognizers show a word after the speaker has finished saying it. Earlybird is a compact recognizer trained to write words from partial audio, even while a word is still being spoken, and to guess the next two words.
In six timed test sets, its median correctly recognized word appeared before the word ended. Among 50 clean-audiobook utterances where all four systems produced output, 68.8% of reference words were correct before ending and stayed unchanged; this count includes transcript words and guesses only when confidence was at least 0.5. Early words can still be revised.
The research
When spoken words arrive only after they end, a voice system must wait for text before responding.
Earlybird tests whether a small recognizer can provide correct words sooner; the results change the expectation that streaming text must trail speech, while leaving room for revisions and weaker performance in noisy conditions.
Earlybird trains a compact streaming recognizer to write words from partial audio, and its median correct word arrived before the word ended on six timed test sets.
The evaluation fed systems the same utterances in 80 ms chunks and measured when each correct word appeared and stayed unchanged, relative to its spoken end.
On clean audiobook speech, Earlybird’s median word arrived 90 ms before that end, while the released Moonshine Tiny runtime’s arrived 270 ms after it.
That timing measure was early for Earlybird on all six timed sets. On the same 50 clean-audiobook utterances where all four systems produced output, 68.8% of reference words were correct before ending and remained unchanged, compared with 3.4% for Moonshine Tiny. These results show that the early median was accompanied by many stable early words in this subset.
The timed tests exclude hardware processing time, and word-end timings from forced aligners can be off by tens of milliseconds. Early words can later be revised, especially on spontaneous or noisy speech; the evaluations are in English.
What the evidence shows
- Words before they end
Earlybird is a compact streaming speech recognizer trained to predict complete words from partial audio. The paper asks whether this can reduce the wait between speech and text without giving the recognizer future audio.
Its training recipe changes what the model is asked to produce at each step, while leaving the starting recognizer’s architecture unchanged.
- Earlier output in context
The Open ASR Leaderboard scores final transcripts by counting word errors against references. On its eight English sets, Earlybird’s mean error rate was 12.6%, close to Whistle’s 12.2%. This comparison describes transcript accuracy, not streaming timing.
- Counting useful early words
A median does not show how many words were useful early. In the same 50 clean-audiobook utterances where all four systems produced output, Earlybird had many more reference words that were correct before they ended and stayed unchanged than Moonshine Tiny. The count includes transcript words and guesses only when confidence was at least 0.5.
- A component with a tradeoff
On these 67 clean-audiobook utterances, the version trained with three extra frames computed from silence after the audio heard so far produced words earlier than the version without those frames, alongside slightly more final-transcript errors.
This comparison suggests a timing and accuracy tradeoff between the two tested versions, but unspecified training differences prevent attributing it to the padding frames alone.
- Accuracy and boundaries
Across eight English sets, the 4-bit Earlybird release had a mean final-transcript word error rate of 12.6%, close to Whistle’s 12.2%. This comparison measures transcript accuracy, not streaming timing, and the mean does not describe every set.
Deep
- Original paper authors
- Sahil Mahendrakar
- Submitted to arXiv
- License
- CC BY 4.0