A speaker linker lets diarizers follow people beyond their slot limit

When a streaming diarizer reuses a speaker slot after someone falls silent, it can keep detecting speech but lose track of who returns. Does that slot limit prevent it from following more people over a session, or only from representing more people at the same time?

The authors add a causal, training-free linker that relabels a frozen diarizer’s speech stretches to track speaker identities beyond its slot limit.

The question is whether a streaming diarizer’s fixed slots limit the total number of people it can follow, or only how many can be represented at once. When a silent speaker’s slot is reused, speech can still be detected while the returning person loses their identity label.

The authors compare a frozen four-slot host with and without a causal linker on sessions above its four-speaker cap. Across 505 such sessions, duration-weighted pooled diarization error rate fell from 36.81% to 22.64%.

Since the linker only relabels the host’s speech, the reduction reflects less speaker confusion; missed and falsely detected speech are unchanged by design.

The evaluation covers eight corpora, but only 53 sessions had more than eight speakers, most from three corpora. The authors make no latency claim, and wrong-identity matching was not tested beyond the 23 speakers seen.