A fixed event budget lets speech codes adapt their timing

If speech codes adapt to sound changes, how can a speech generator still know how many codes an utterance will require? Threshold-based codecs let content change the count, making sequence length harder to predict from audio duration alone. The paper asks whether timing can adapt while count remains tied to utterance length.

RubatoCodec fixes how many speech events an utterance receives while adapting their boundaries to changes in the sound, enabling incremental decoding.

Can a speech codec adapt its timing to changing sound while keeping the number of codes predictable from utterance length? Threshold-based allocation can make the count vary even at a fixed setting. RubatoCodec sets an event budget from utterance length and adapts where events begin.

On English read speech, its adaptive architecture had lower transcript word error than a fixed-rate codec at a nominal 12.5 events per second. The transcript measure compares automatic recognition of reconstructed audio with the original.

A separate placement control found mixed results across quantizer depths, indicating that adaptive timing can help but does not win in every tested setting.

The main architecture comparison changes event aggregation, the time grid, and decoder upsampling together, so it cannot show that adaptive placement alone caused the improvement. Comparisons use one trained model per configuration; the placement test uses one seed and English read speech.