A pilot helps allocate compute when verifiers err
Cette page n’a pas encore de traduction vérifiée. La version anglaise est affichée. Lire la version anglaise
En termes simples
Language models can spend extra computing time by making several answers and asking a checker to choose one. But the checker can make mistakes, and those mistakes may repeat. So making more answers does not always mean the submitted answer is more likely to be right.
SPLIT learns from a small set of labelled tasks how answers and checks behave, then chooses how to divide a fixed budget. In the reported tests, its choices came close to the best choices identified after the test tasks were known.
La recherche
When a model can generate several answers, how should a fixed computing budget be divided between making candidates and checking them? The challenge is that correct answers must be generated and then selected, while real verifiers can make correlated mistakes. More candidates or more checks may not improve the final answer.
SPLIT learns from a small labelled pilot how task difficulty and verifier errors interact, then allocates a fixed compute budget to maximize predicted success.
The researchers calibrated SPLIT on one random half of the tasks and tested its choices on the other half. They compared each choice with the best allocation in hindsight on those evaluation tasks, measuring the difference in verified success. Across settings, SPLIT variants stayed within 0.8 percentage points of that reference.
This supports the claim that a small pilot can guide allocation close to the hindsight choice in the tested settings. The best variant differed across settings, and the reported range combines multiple tasks and verifiers.
The planner’s estimates assume the labelled pilot represents the tasks where it will be used. The reported same-agent software-agent selection tests reached only 16 rollouts per issue, so the study does not establish how selection works at longer horizons.
Analyse
Détails
- Auteurs de l’article original
- Srinivasan Iyengar
- Soumis à arXiv
- Licence
- CC BY 4.0