MagicBeanStalk builds training problems from hard seeds

When a model gets every sampled answer to a question right, or every answer wrong, those responses offer no contrast for learning from that question. The paper asks whether hard questions can still help by revealing reasoning steps that can be turned into new, more useful training problems.

MagicBeanStalk turns hard problems into new, model-calibrated training questions by rebuilding their verified reasoning steps.

The authors ask whether a hard problem can guide the creation of a new one that gives a model useful feedback. A test on 98 math seeds found that a verified example of a simpler, blueprint-based subproblem helped the target model solve some original problems.

MBS applies that idea recursively, then screens candidates with the target model. After continual training, gpt-oss-120b’s mean pass@1 accuracy across eight math tests rose from 61.2 to 69.8; training on the full source mixture reached 62.3.

These results come from particular models, problem sets, and response limits. Generated data and the models used to audit it may contain errors; the finance evaluations do not establish that such training data is reliable for financial decisions. Some teacher models are proprietary APIs, so changes to them may limit exact reproduction.