Two inference changes reduce long-conversation failures
Why Full-Duplex Speech Models Collapse in Long Conversations · arXiv:2610.32349
In plain words
Full-duplex speech models listen and speak at the same time, but their behavior over long conversations has been hard to measure. This paper introduces a test suite with recorded conversations and sessions with a responsive spoken partner. In tests of five models, each eventually degraded or ran out of context.
The authors also developed two changes that work during generation without retraining: one keeps the first piece of the model’s memory, and the other discourages repeated words. Together, they reduced measured failures in two tested models, though one still sometimes went silent.
The research
Speech models that listen and speak continuously can have their generated output enter the next input and influence later speech while their stored context changes.
Can they remain stable over conversations lasting many minutes, and can simple changes made during generation reduce the failures that appear?
The authors introduce a long-conversation test and two training-free inference changes that reduce measured failures in two sliding-window speech models.
The Full-Duplex Long-Horizon Benchmark tests whether speech models that listen and speak at once can remain stable over long conversations. The authors evaluated five models, then applied two inference changes to the two sliding-window models, whose memory discards older entries as new ones arrive.
In fixed replays, the changes reduced repetition, silence, and gibberish, though PersonaPlex still had prolonged silence in some runs. In adaptive sessions, Gemini-2.5-Flash rated one-minute speaking bins for fluency and sense and for responsiveness to the partner; mean baseline scores fell to the lowest rating within about ten minutes, while scores with both changes stayed steady through sessions of up to 20 minutes. Together, these results support the changes for the tested models and settings.
The diagnosis and fixes were tested on two sliding-window models, and the benchmark observed systems for finite periods. Its failure measures and evaluator scores do not establish overall conversation quality or indefinite stability; PersonaPlex still had prolonged silence in some replays.
What the evidence shows
- A longer test for speech
The paper introduces the Full-Duplex Long-Horizon Benchmark (FD-LHB), a test suite for whether speech models that listen and speak at once remain stable over long conversations. Existing tests described by the authors covered a turn or a few rounds.
FD-LHB instead includes fixed replays of recorded speech and interactive sessions with a responsive spoken partner, so it can examine both controlled and adaptive conversations.
- Five models degrade or stop
In FD-LHB, the authors evaluated five models using sliding-window or growing-context memory. Each model eventually showed repetition, prolonged silence, or gibberish, or reached its context limit.
The distinction matters: sliding-window models degraded without exhausting their memory, while growing-context models could degrade before reaching capacity. The finding establishes a long-conversation problem in these tested systems, not a general rule about all speech models.
- Different failures, different clues
To investigate silence and gibberish, the authors studied Moshi and PersonaPlex, both sliding-window models. As they increased the attention-window length, silence and gibberish began later; gibberish also followed this pattern with silence-only input.
Repetition did not show a consistent relationship to window length. The authors interpret the contrast as evidence linking silence and gibberish to cache turnover, while pointing to a separate explanation for repetition.
- Two diagnostic signals
The authors tested whether a particular cache entry mattered by keeping one early entry while changing which one it was. In the tested runs, retaining the first entry kept predictive entropy below the H=2 threshold every time; retaining none did not, and other single entries failed in at least one run.
This diagnostic measures uncertainty about the next output, not conversation quality. Separately, inserting a previously absent word more often increased its later recurrence in both models, especially PersonaPlex.
- Inference changes reduce failures
The authors introduced Initialization-Anchor Pinning, which keeps the first cache entry, and Duplex Repetition Penalty, which discourages recently used words and guards against prolonged silence. In matched Moshi replays, the two methods showed complementary effects: IAP alone eliminated gibberish, while DRP alone eliminated repetition but left gibberish in every run.
Combined, they had no repetition or gibberish and prolonged silence in three of 40 runs. Results for PersonaPlex also improved, but prolonged silence remained more common.
Deep
- Original paper authors
- Yeonju Kim, Bella Godiva, Se Jin Park, Yong Man Ro
- Submitted to arXiv
- License
- arXiv non-exclusive