Open-vocabulary speech runs on a low-cost microcontroller

Low-cost devices face a choice: fixed commands keep speech local, while broader recognition can require sending audio to the cloud or fitting a large model into a tiny chip. Can an accurate open-vocabulary recognizer fit locally without an accelerator, and what does that require?

Oído combines integer speech recognition with memory-aware scheduling to run open-vocabulary English speech on an ESP32-S3, though not yet in real time.

The authors compared Oído’s int8 inference with the same model in full precision on the complete LibriSpeech test-clean and test-other speech sets. The test measures the percentage of words transcribed incorrectly: Oído scored 3.70% and 8.23%, versus 3.68% and 8.11% for the reference.

In the reported ESP32-S3 comparison, Oído’s host build used the same int8 arithmetic as the device and reported fewer errors than Espressif’s vendor-reported MultiNet7: 3.70% and 8.23% versus 8.5% and 21.3%.

MultiNet7 uses a fixed command list and its evaluation protocol is not detailed, so this supports an on-chip comparison under stated conditions, not a universal ranking.

The accuracy figures come from the host build, while the physical-board study used one board and one-core execution.

For the int8 CTC model with language-model decoding, the mean real-time factor was 1.97 across 27 clips; the two-core schedule corrupted results, so feasibility here does not establish real-time or broadly validated deployment.