Training logs guide when multimodal tasks are replaced
A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement · arXiv:2610.38568
In plain words
A-RSI is a way to continue training a model on picture questions without needing people to write the answers. Programs make each picture and question and calculate its answer from the same data.
The method watches training logs and replaces the current set of questions when the logs meet a rule. In one Qwen model run, its score across eleven tests rose during eight more training rounds. A comparison favored using new questions over reusing the old set, but the new set was also smaller.
The research
A model can spend more training on a task pool after its learning signal has declined, yet replacing tasks too soon consumes new tasks unnecessarily. How can training feedback guide that decision while keeping answers to generated questions checkable?
A-RSI uses training logs to decide when to replace multimodal tasks, with program-computed answers for the new questions.
A model may keep seeing little new learning signal from its current task pool, but replacing that pool too soon uses fresh tasks unnecessarily. Can training feedback guide replacement while keeping answers checkable?
A-RSI watches training logs and activates prepared tasks whose answers programs compute from the same data used to make each question. On tests of multimodal question answering, one Qwen run’s equal-weight average accuracy across eleven test sets rose from 57.77 after initialization to 61.05 after eight continuation windows.
At matched progress, fresh-pool training scored above continued reuse, supporting the combined approach in that run.
The main Qwen results come from one training run over eight continuation windows. The matched-progress comparison changes pool contents and size together, and the tested results do not establish that adaptive timing itself improves scores or that the gains continue beyond this setting.
What the evidence shows
- Two questions in continued training
A-RSI is a method for continuing multimodal reinforcement learning: it supplies new image-based questions with answers computed by programs, and uses training logs to decide when to replace the active task pool.
The research addresses two linked questions: when existing tasks may be providing less useful learning signal, and how to give generated tasks answers that do not depend on the model’s own uncertain votes.
- Answers come from programs
Each task program samples one underlying configuration, then uses it to create an image and question and compute a reference answer. The learner receives only the image and question.
This makes the answer independently checkable against the program’s calculation, though checks do not prove every rendered task is correct or unambiguous. An audit found agreement on all 1,000 sampled tasks across five generator groups, not every task family.
- Logs guide pool changes
The controller watches two training-log signals: how many distinct prompts produce reward differences among responses, and how often logged responses are correct. After a minimum observation period on the same pool, a decline in the first signal or a sufficient rise in accuracy triggers replacement.
These are practical indicators, not proof that the model has mastered its pool or stopped improving.
- Fresh pools help in one run
The main tests measure multimodal question-answering accuracy across eleven full test sets. Their Macro11 score gives each test set equal weight.
In one Qwen3-VL-8B-Instruct run, Macro11 rose from 57.77 after initialization to 61.05 after eight continuation windows using A-RSI. At matched training progress, fresh-pool training scored 60.40 versus 58.35 for continued reuse, supporting the combined replacement approach in that run.
- Timing remains unresolved
The matched-progress result combines changes in task instances and pool size, and both training arms were run once. A separate comparison using the same frontier task selection found no accuracy advantage for detector-triggered timing over refreshing every two windows.
The experiments therefore support the observed Qwen gains from the implemented fresh-pool approach, but do not show that adaptive timing caused them.
Deep
- Original paper authors
- Yu Liu, Youxing Li, Yuedong Tan, Zhiwei Yang, Zihan Su
- Submitted to arXiv
- License
- arXiv non-exclusive