Two adapters let audio–video generation stream in a few steps

Can a joint audio–video generator learn to produce output in chunks and use only a few generation steps per chunk without one training stage undoing what another learned? Sequential training changes the model’s weights at each stage, so the next stage must adapt to that changed starting point.

The method trains a streaming adapter against unchanged weights, then adds it to a separately obtained few-step adapter during generation.

With a frozen MiniMax-H3 backbone, the authors train a streaming adapter for chunk-by-chunk output and combine it during generation with a public adapter that uses four denoising steps.

They compare this composition with two sequential training routes on unseen prompts, scoring each clip against the backbone’s own version of that clip.

On image quality, the composition scores 0.914, close to 0.915 for a bidirectional four-step system and above 0.854 and 0.877 for the sequential routes. This supports the claim that the capabilities can be combined without sequential training while retaining a similar appearance score in this test. Lip–speech alignment, however, favors one sequential route.

The held-out image-quality scores compare each output with the backbone’s own generation of the same clip, not with real source video. The study uses talking-head footage and one audio–video backbone, so transfer beyond that setting remains untested.