Video compression cuts generation time while preserving Wan2.1 scores

How can a video generator process fewer latent tokens without losing its ability to make good videos? Stronger compression reduces the work in the generator, but can erase details and shift the representation away from what its pretrained generator learned.

Reconstruction quality alone may not show whether the generator can use the compressed representation.

GRACE compresses a pretrained video autoencoder, aligns its representation for the generator, then adapts the generator to denoise the base ahead of the residual in the same pass.

On Panda-70M reconstruction, the single-latent baseline scored better than GRACE-VAE. But after each latent was paired with an adapted Wan2.1 generator, it scored lower on VBench-I2V, a test of image-to-video generation that scores dimensions of video quality.

Better reconstruction alone did not predict better generation in this comparison. On VBench text-to-video and image-to-video tests at 480x832 resolution with 81 frames, GRACE used about one-eighth as many latent tokens as uncompressed Wan2.1 while its task totals stayed close to or above Wan2.1's. This supports the method in the reported setting: fewer tokens coincided with similar measured video quality.

These results come from the reported Wan2.1 settings. Compression can lose small details such as distant faces, sign text, and thin structures, and applying GRACE to another pretrained pipeline requires training a new autoencoder.