In plain words
Many systems need to choose from a list of answers, and the list can change with each request. This paper introduces Crivo, a family of compact encoders that gives each allowed answer a probability.
Its released models read all choices together; the authors also tested a version that rereads the request separately for each choice.
In a 32-million-parameter comparison, separate reading scored higher on a sample of decision tests when both versions started from a reranker. The advantage was not clear when they started from a raw encoder, so the result depends on how the model began.
The research
When a model must choose among answers that can change from one request to the next, should a compact encoder compare the choices together or score each one separately? The joint approach uses fewer encoder passes, but separate reading resembles the paired inputs used to train rerankers. The paper tests whether that difference matters.
The paper introduces Crivo, a family of compact decision encoders, and compares scoring all choices together with rereading the request for each choice.
The authors compared two ways for a compact encoder to score a changing list of answers: process all choices together, or reread the request for each choice.
In a 32-million-parameter study, both readers used the same training subset and objective; the joint reader was trained for as many steps by repeating rows.
On a reserved sample of decision tests, the per-option reader had higher chance-corrected skill than the joint reader in this approximate compute-matched comparison.
A separate initialization control found no clear reader advantage from a raw encoder, so the main result is specific to the reranker starting point and does not establish a general benefit of separate reading.
The reader comparison used one 150,000-question subset at one model size, 32 million parameters. Its compute-matched joint reader repeated training rows, and the study did not test the per-option reader above that size or with the full Crivo training recipe.
Analysis
Deep
- Original paper authors
- Tardelli Ronan Coelho Stekel
- Submitted to arXiv
- License
- arXiv non-exclusive