One language model learns retrieval and answer generation

Can one language model represent a query well enough to find relevant documents and still use it to generate an answer? The final query state is involved in both jobs, even though retrieval needs a globally comparable representation and answer generation needs states that predict the next words.

ULME trains separate retrieval and answer readouts in one causal model, linking them by teaching a query-side slot to match a summary of the answer.

The authors tested whether one model could retrieve relevant documents and generate answers without sacrificing either capability.

ULME uses one checkpoint for both jobs, with learned slots separating the query’s retrieval representation from answer prediction and a training signal that links the query to the answer.

On three Qwen3 sizes, ULME scored above LLM2VEC-GEN on general text-embedding and reasoning-focused retrieval tests. Against a generation-only fine-tune from the same backbone and generation data, it was generally at least as strong on the reported generation tests. The results support joint performance in these evaluations.

These results come from one final checkpoint per configuration, with the main tables showing three Qwen3 sizes. The authors also report preliminary qualitative degradation in some chat, formatting, and instruction-following behaviors; they do not quantify its extent.