A method steers answers as it revises question context
Cette page n’a pas encore de traduction vérifiée. La version anglaise est affichée. Lire la version anglaise
En termes simples
EMO is a method for helping a frozen language model answer one question at a time. After each answer, an outside judge gives feedback without seeing the correct answer.
EMO uses that feedback to rewrite the instructions for the question, then directly adjusts the model’s next-word choices as it answers again.
Across eight models and five reasoning-task groups, EMO scored above zero-shot answers in 39 of 40 comparisons. That overall result does not mean every model or task improved.
La recherche
Can feedback on a model’s answer do more than rewrite its instructions? The paper asks whether it can also directly control how the next answer is generated. Context updates already change token preferences, but ordinary generation offers no separate control over the strength of that change.
EMO revises a question-specific context from judge feedback, then uses that change to steer token choices as a frozen model answers again.
The researchers tested eight models on six reasoning tests, combining the two AIME editions into one task group. Across 40 model–task-group pairs, EMO’s rule-scored accuracy exceeded zero-shot accuracy in 39, with a mean gain of 9.10 percentage points.
Correctness was checked against answer keys using task-specific rules, while a separate judge guided revisions without seeing those answers. This supports a gain from the full EMO procedure under greedy decoding.
The proof guarantees a one-round reduction in target KL divergence only if the context update improves that measure and the actual steering direction aligns positively with the target; the steering strength must also be sufficiently small.
It does not guarantee higher accuracy, improvement across rounds, or transfer beyond the evaluated reasoning tests and two model families.
Analyse
Détails
- Auteurs de l’article original
- Wei Xia, Yehui Tang, Zhi-Hong Deng
- Soumis à arXiv
- Licence
- CC BY 4.0