One model learns faster image generation and understanding
In plain words
One model can create images and answer questions about images, but it usually needs many rounds of token predictions.
Omni-Diffusion-Distill is a training method that teaches such a model to finish in fewer rounds, then trains it on its own outputs to address repeated words and overly certain image predictions.
On five text-to-image tests, its eight-step version led the other distilled methods. The study tested the method on one model, so the result does not establish performance on every multimodal model.
The research
Can one multimodal model generate images and understand them while using fewer rounds of token prediction? Cutting rounds can make parallel text predictions repeat and can make image predictions too certain, so reducing computation while preserving both abilities is a specific challenge.
Omni-Diffusion-Distill teaches one discrete text-and-image model to skip decoding steps, then refines its own outputs to limit repetition and image overconfidence.
The authors first train the student on cached teacher outputs so it can skip decoding steps, then refine it on states from its own outputs. Text and image tasks share this process, with a repetition penalty for parallel text and adjusted image guidance to preserve uncertainty in image predictions.
On text-to-image tests, the student led distilled methods at eight image-decoding steps. On COCO captions, GPT-5.5 rated its 64-step captions 57.2 out of 100, versus 28.4 for the teacher at the same budget, using accuracy, completeness, and coherence. These results show performance across image generation and captioning on the tested model.
The authors validate the method on one model, Lumina-DiMOO. The student still trails the full-step teacher on some tasks, especially text rendering and detailed captioning, and trails the few-step teacher on image editing at eight steps.
Analysis
Deep
- Authors
- Hong Huang, Chenhongyi Yang, Junzhe Sun, Animesh Sinha, Wuyang Chen
- License
- arXiv non-exclusive