A small decoder takes recommendation search off the full model
Separating Thinking from Spelling for Efficient Generative Recommendation · arXiv:2610.32980
In plain words
Recommendation systems use a person's past choices to suggest what they may want next. But a large model can spend much of its time repeatedly searching through possible item codes.
Deco has the large model summarize the history once, then gives the search to a small decoder. Across seven public datasets, it matched or beat its full-model comparison on recommendation accuracy measures. This shows the approach worked in those tests; it does not establish results for every setting.
The research
When a recommendation model has already compressed a user's history, does the full model still need to do the repeated work of searching through candidate items? The paper measures that remaining cost and asks whether a smaller decoder can generate candidates while retaining recommendation quality.
Deco has the backbone summarize a user's history once, then lets a small decoder generate recommendation candidates from that summary.
Deco has the backbone read a user's interaction history once and form a compact concept of plausible next items. A small decoder uses that concept to generate candidate codes.
The authors compared this design with FusedGR, which uses the full backbone during candidate generation, on seven public datasets with a global temporal split.
Deco matched or exceeded FusedGR on recommendation accuracy measures averaged over test users and three training seeds. In serving tests on one H100 GPU, it served 3.2–7.3 times as many users per second across the datasets while meeting a 50 ms 95th-percentile latency target.
Together, these results show that the tested decoder handoff retained quality while increasing serving capacity under that constraint.
The evidence covers seven public datasets under a global temporal split, with the serving comparison run on one H100 GPU using beam width 100 and a 50 ms p95 latency target. The separate leave-one-out comparison was limited to Beauty.
What the evidence shows
- A remaining search cost
Deco, or decoding from concepts, is a generative recommendation method. It addresses a cost that remains after a model compresses a user's interaction history: the full model may still repeatedly process candidate-generation steps.
The research asks whether a smaller decoder can take over those steps while retaining recommendation quality.
- A compact handoff
The method has the backbone read the history once and produce a compact concept representing plausible next items. A small decoder uses that concept and the codes already generated to produce candidate semantic IDs, with a trie restricting the next code to valid choices. The backbone therefore does not repeat the search for each candidate.
- Search dominates one test
The authors first measured where generation time went. On BPC-23 at batch size 128, beam search accounted for 91% of FusedGR's generation time, even with a shared history cache. This identifies why shortening history input alone left work to remove in that measured setting.
- Quality and serving results
On seven public datasets using a global temporal split, Deco matched or exceeded FusedGR on the reported recommendation accuracy measures. The main results average over test users and three training seeds.
For serving, on one H100 GPU at beam width 100, Deco served 3.2–7.3 times as many users per second across those datasets while meeting a 50 ms 95th-percentile latency target.
- A repeat-heavy exception
The compact summary was enough on the tested low-repeat Sports and BPC-23 datasets: giving the decoder richer access to history changed accuracy measures little and could increase latency. Yambda qualified that result.
A pointer to past tracks raised Deco's NDCG@10 from 0.079 to 0.171, consistent with the authors' interpretation that exact repeats need access to item identities.
Deep
- Original paper authors
- Danil Gusak, Anna Volodkevich, Ivan Oseledets, Evgeny Frolov
- Submitted to arXiv
- License
- CC BY 4.0