Monsoon builds speech data that improves conversational recognition

When speech-recognition systems are trained on limited or mostly read-aloud data, do they perform well on unscripted conversation?

The authors examine whether collecting and verifying spontaneous speech can address that gap, and whether the resulting models hold up on speech gathered independently of the training collection.

Monsoon collects and verifies spontaneous conversations, then selects speech segments to train recognition models for languages with limited data.

The authors compare Whisper medium trained further on each Monsoon corpus with IndicWhisper, which has the same architecture but a different training recipe. On Vistaar, whose speech comes from IndicWhisper’s training sources, results are broadly similar.

On independently collected conversational tests, Monsoon-trained Whisper has a 20–44 percentage-point advantage in semantic word error rate, a measure where a language-model judge discounts transcript differences that preserve spoken content, across reported language-level results.

This contrast supports the view that Monsoon data helps with conversation beyond matching the public-data model on familiar sources. The authors link the gap to a mismatch between read speech and spontaneous speech, but the comparison does not isolate that cause.

The IndicWhisper comparison excludes Kashmiri because that model does not support it. Since the models’ training recipes also differ, the comparison cannot attribute the conversational-speech advantage to training data alone.