APEX uses one network to generate and estimate streaming video
Few-Step Streaming Video Generation via Condition Shifting · arXiv:2611.14033
In plain words
APEX is a method for generating long videos in a few steps. It uses one network in two ways: ordinary text guides video generation, while shifted text makes that network estimate the distribution of the videos it generates. A separate frozen teacher still helps guide training.
In tests of 180-second videos, APEX used less peak GPU memory and took less time per training iteration than DMD, which uses a separate estimator. Expert ratings favored APEX for following instructions but favored S2-Avatar for motion quality.
The research
A separate estimator adds memory and computation during video-generator training, and it must keep up as outputs change. APEX tests whether sharing those roles can reduce that burden while retaining the teacher, with reported results that also include a motion-quality trade-off.
APEX is a few-step streaming-video method that uses shifted text conditions to make one network generate video and estimate the distribution of its own outputs.
APEX uses the original text condition to generate video and a shifted condition to estimate its outputs in the same network; a frozen teacher still provides reference guidance. The evaluation compared 180-second APEX and S2-Avatar videos using three backbone evaluations per block and a shared one-step spatial Refiner.
On 160 Progressive cases, the benchmark’s Gemini judge gave APEX a higher mean instruction-fulfillment score. Its visual-aesthetics mean was also higher across all 320 cases, while expert raters favored S2-Avatar on motion quality. The findings support gains on specific measures, not across-the-board superiority.
The evaluations cover 180-second videos under the reported generation settings, and the human ratings cover 70 paired image-to-video cases. The theoretical analysis does not establish long-horizon stability or an exact KL gradient for the full video trajectory.
What the evidence shows
- One network takes both roles
APEX is a method for few-step streaming video generation that uses shifted text conditions to make one network generate video and estimate the distribution of its own outputs. This addresses a cost in Distribution Matching Distillation, where a separate fake-score network adds training memory and computation and must track changing outputs. APEX retains a frozen teacher for reference guidance.
- A small lead on instructions
StreamAV-Bench tests how well videos follow their instructions. On its 160 Progressive cases, the Gemini judge’s mean score was slightly higher for APEX (3.856) than for S2-Avatar (3.831), with the same generation budget and Refiner.
- A pooled aesthetics advantage
The LAION Aesthetic Predictor scores visual aesthetics. Across all 320 Progressive and Interactive cases, its pooled mean was higher for APEX (0.641898) than for S2-Avatar (0.632606); this average does not mean APEX scored higher in every case or on every quality measure.
- Expert ratings are mixed
In 70 paired image-to-video cases, experts’ instruction-following ratings favored APEX (+30.0% net preference), while their motion-quality ratings favored S2-Avatar (−8.6%).
Net preference subtracts losses from wins and divides by all pairs, including ties, so the judgments show a trade-off between these dimensions.
- Training costs and limits
Under matched training settings, APEX’s mean iteration time was shorter than DMD’s (32.6 seconds versus 35.9 seconds), while APEX retained the frozen teacher. This comparison is consistent with a training-cost benefit from sharing the estimator, but its reported scope is those matched settings.
Deep
- Original paper authors
- Deyuan Liu, Jungang Li, Kaiwen Zheng, Kai Jiang, Jintao Chen
- Submitted to arXiv
- License
- CC BY 4.0