A sparse weight repair improves math answers from a looping model
Tree-Ising: Repairing Language Models with Bit-Level Ising Edits · arXiv:2611.14210
In plain words
Some small reasoning models get stuck repeating text and use up their answer limit before finishing a math problem. Tree-Ising is a method that changes a small set of model weights, using clues from the model’s own answers to choose changes without training it.
On 500 math problems, the repaired Qwen3.5-0.8B model had higher average accuracy than a version repaired by an AI agent. The comparison used the same evaluation setup, but the edits were not matched in size.
The research
A model that spends its answer budget repeating text may have the ability to solve a problem but fail to finish.
A direct weight repair could change how we judge that failure: on the tested math tasks, the model’s answers improved after a small set of weights was edited, while the edit also changed some previously correct answers.
Tree-Ising uses forward-pass evidence to select sparse, format-preserving weight edits, then tests whether they reduce reasoning loops and improve math answers.
On MATH-500, the evaluation compared Tree-Ising with an agent-made repair using the same harness and random seeds. Tree-Ising’s 54.6% mean accuracy exceeded the agent edit’s 51.7%, based on four generated answers per problem graded for correctness.
The comparison supports the repair’s effectiveness, with a qualification about how it was designed and calibrated.
The Qwen3.5-0.8B recipe was designed after examining an earlier agent’s public logs, reused its chosen layers, and used a larger edit dose. The evaluations cover two models from one family on math tasks, with one generation seed per benchmark run.
What the evidence shows
- A direct repair improves answers
The same-harness comparison favors Tree-Ising over the agent-made repair on Qwen3.5-0.8B: mean correctness across four answers per MATH-500 problem was 54.6%, versus 51.7%.
Because Tree-Ising used a larger edit dose and its recipe reused the agent’s selected layers and public logs, this does not show which repair would win if discovered independently at equal cost.
- Successful conclusions matter
On a practice screen of 128 problems with one answer each, removing the successful-conclusion signal coincided with lower accuracy than the selected setting. More answers also hit the token cap, suggesting that this signal matters to the tested recipe. The screen is too imprecise to establish the size of this effect on the full evaluations.
- The gap persists on unseen problems
The advantage over the agent edit also appears on the 400 MATH-500 problems not used during development: mean correctness across four answers per problem was 53.8%, versus 50.8%.
This supports the comparison beyond the problems encountered during development, while remaining a subset result for Qwen3.5-0.8B on the same math test.
- A second model shows transfer
On GSM8K, a test of grade-school math word problems, accuracy is the share of problems answered correctly. With greedy generation in thinking mode and an 8,192-token cap, Qwen3.5-2B rose from 21.6% accuracy before the edit to 69.4% after it across all 1,319 problems.
This shows the recipe was applied without a reference agent repair in this setting; the edit also changed some answers the base model had solved correctly.
Deep
- Original paper authors
- Hadi Salloum, Ammar Ali
- Submitted to arXiv
- License
- arXiv non-exclusive