A language model ranks pages from search clicks
Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages · arXiv:2610.35256
In plain words
Some searches are looking for one particular page. A click table can miss a new way of asking, while a search system based on similarity can put a related page first.
Octen-WebGR trains a language model to produce a short identifier for the page people clicked, then looks up its URL. On later traffic, its 4B model ranked the target first for 59.6% of navigational clicks, compared with 46.5% for a similarity-based system adjusted using click counts. This was an offline test, not a live search experiment.
The research
When someone searches for a particular page using words absent from past searches, a click table cannot answer directly, and a similarity-based system can rank a related page first. Can a search method learn which destination people mean while restricting its answers to pages that exist?
Octen-WebGR trains a language model to generate page identifiers from clicks, then looks them up to rank existing URLs for navigational searches.
Octen-WebGR learns to generate a unique identifier for the page associated with a query, then maps that identifier to an existing URL. The authors test whether it ranks the most-clicked target first on September traffic, after training data through August.
The 4B model ranked the target first for 59.6% of navigational clicks, compared with 46.5% for dense retrieval adjusted by training click counts. This supports the method on that offline, click-weighted comparison; results vary by how many training examples the target page has.
These are offline results from the paper’s traffic samples and training data. The matched scaling comparison covers 8 million and 32 million URLs, and the spread test covers pages with at least 16 training pairs; neither rules out capacity limits at larger scales.
What the evidence shows
- The page must rank first
Navigational search has a specific goal: put the intended page first. A click table can return a page for a query it has seen, but has no entry for a new query.
Dense retrieval can handle new wording by comparing query and page representations, yet a similar page may rank above the intended one. The paper asks whether a language model can learn the link between queries and destination pages from clicks.
- Generate a page identifier
Octen-WebGR is a generative retrieval method: it trains a language model to produce a unique pair of codes for the page associated with a query. A table maps each valid pair back to a URL, and constrained decoding restricts answers to identifiers in that table.
The released 4B model covers about 201 million URLs. The evaluation checks whether systems rank the most-clicked page first, with September traffic occurring after the model’s training data.
- Compare on later traffic
On that later traffic, the 4B model ranked the target first for 59.6% of navigational clicks, compared with 46.5% for dense retrieval reranked using training click counts. The comparison uses the same traffic, URL universe and click-weighted measure.
The result shows stronger offline ranking against this baseline, but the target is defined by clicks and the experiment does not test a live search service.
- The advantage has limits
The advantage depends on how much training data a page has. For new queries about familiar pages, with at least 11 training pairs, Octen-WebGR outperformed dense retrieval with its click-count adjustment.
For cold pages with 1 to 10 pairs, dense retrieval without the adjustment ranked targets first more often. The authors’ separate training analyses link retention to when examples appear and the learning rate at exposure.
In a separate test on pages with at least 16 training pairs, spreading examples across queries improved hit@1 on held-out queries compared with repeating two queries. This supports better generalization to held-out queries in that tested group, not a retention claim.
Deep
- Original paper authors
- Guangwei Xu, Wenkai Zhang, Yawen Liu, Dongxu Zhang, Xijie Xu
- Submitted to arXiv
- License
- arXiv non-exclusive