One image model handles creation, editing and references
WeLM-Image: A Unified Image Generation Model towards Visual Intelligence · arXiv:2611.07392
In plain words
Image models need to follow detailed prompts, change only what an editing instruction asks for, and use several reference pictures without mixing up their contents. WeLM-Image is one model for creating images from text, editing images, and combining multiple references.
It separates the work of understanding words and pictures from making new image content, while letting those parts share context. On a test where evaluators judged generated images for subjects, style, and composition, it ranked first among the open-source models compared.
The research
The question is whether one model can keep instructions and image references straight while creating or editing pictures. A unified system makes those capabilities part of the same model, and the PRISM-Bench result offers one concrete measure of its image-generation quality. Other tasks still need their own evidence.
WeLM-Image is a single image model that shares context between understanding and generation paths to create images, edit them from instructions, and use multiple references.
To test one part of this capability, vision-language-model evaluators used PRISM-Bench to judge generated images for imagination, accurate subjects, style, and composition. WeLM-Image scored 7.56 overall and ranked first among the open-source models included in that test.
That result supports a focused conclusion about image generation on PRISM-Bench. The report also evaluates text rendering, editing, and multiple-reference composition, but those results address different tasks.
PRISM-Bench measures generated-image quality on one test. WeLM-Image currently focuses primarily on image generation and editing; stronger visual reasoning and more reliable control under complex instructions remain future work.
What the evidence shows
- One model, several image tasks
WeLM-Image is a unified image foundation model for creating images from text, editing them from instructions, and combining multiple references. The problem it addresses is fine-grained control: following detailed prompts, preserving content that should not change, rendering text accurately, and keeping the roles or attributes of references distinct. The model brings these tasks together in one system.
- How the parts share context
The model represents instructions, image references, and target image content as ordered segments in one sequence. Separate parameter paths handle understanding information and image generation, while shared attention lets them exchange context.
Task-specific masks control which segments can interact. This arrangement is the architectural answer to the challenge of using language and visual references together during image creation and editing.
- Training toward one model
Training first mixes understanding with image creation and editing, then increases resolution and shifts toward generation, editing, and multiple-reference examples. Separate reinforcement-learning branches later refine image creation and editing from a shared checkpoint.
On-policy distillation combines those branches into one model. The training design therefore pairs a common backbone with separate refinement for the two image tasks.
- A bounded generation result
PRISM-Bench tests images generated from text prompts. Vision-language-model evaluators assess imagination, subject rendering, style, and composition; WeLM-Image scores 7.56 overall and ranks first among the open-source models included in that test.
The result supports a claim about this specific generation evaluation, while the report’s other tasks require separate evidence.
Deep
- Original paper authors
- Canmiao Fu, Chen Li, Guohui Zhang, Hao Lu, Hongyang Wei
- Submitted to arXiv
- License
- CC BY 4.0