A text-based method makes audio answers traceable to named sounds

When an audio model gives a reason for an answer, can we tell whether that reason reflects what it heard? Typical systems generate their reasoning inside the same model that answers, so a readable explanation alone may not reveal which audio evidence shaped the decision.

Listen-to-Reason routes audio into readable concept nodes, then lets a text-only language model answer from those nodes and a speech transcript.

The method routes each audio clip through small components that select named sound or speech concepts, then gives those concepts and a speech transcript to a text-only language model.

On SAKURA, a test of single- and multi-step questions about gender, language, emotion, and animal identity, the L2R reader answered 80.1% of 3,997 questions correctly, versus 73.3% for the strongest compared audio model, which heard the clip.

The shared questions, options, and scoring make the results comparable on that test, while differences in architecture, training, and input mean the comparison cannot isolate the effect of readable evidence.

Separate interventions on SAKURA single-hop questions also support a role for the nodes: changing the deciding node often changed the answer in the expected direction.

The SAKURA comparison does not isolate the effect of readable evidence: the systems differ in architecture, training, and input. The domain-addition results cover species identification, while the current tree cannot express several event-level or scene-level details.