A harness improves distributed model training speed
InfraRSI: Harnessing Recursive Self-Improvement for Training Infrastructure · arXiv:2611.11558
In plain words
The paper introduces InfraRSI, a harness that helps improve the settings and code used to train large language models across many GPUs. It tries changes, checks that each run did the required work and stayed within numerical limits, then verifies promising results against a reference run.
Across four dense and mixture-of-experts workloads, it achieved higher combined speedups over three comparison systems, though the average does not describe every workload.
The research
A reported speedup is useful only if the run did the same training work and the timing survives noise.
InfraRSI makes those checks part of optimization, so its gains on the tested workloads are more credible than a raw timing claim; the results also suggest that tuning source code alongside settings can matter.
InfraRSI is a five-stage harness that searches distributed training settings and code, verifies candidate runs, and reuses measured experience across GPU types.
For each task, InfraRSI searches launch settings and code patches under a fixed contract for the work and allowed numerical behavior, then reruns leading candidates against the reference. Across four dense and mixture-of-experts workloads, its geometric mean of speedups was 1.83x, compared with 1.37x for RE-Bench, 1.34x for Auto-Research, and 1.23x for settings-only OptiCo.
The comparison supports its combined search on these workloads, though the methods kept native search rules.
The four-workload comparison held task contracts constant, but each baseline used its native search rules and shared starting configurations were not specified. Experience transfer was tested on one target workload and GPU setup.
What the evidence shows
- Search settings and code
InfraRSI is a training-optimization harness that searches launch settings and source code together, then checks candidate runs before promoting changes. That combination matters because settings and code can interact, while memory limits, compiler conflicts, timing noise, and numerical drift can make an apparent gain unreliable.
The paper's central claim is that coordinated search with verification can improve training speed on the workloads it tested.
- Measure instead of guessing
The harness also accounts for the limits of language-model estimates. Across reported cases, zero-shot speedup predictions had Spearman rank correlations from −0.49 to −0.06 with measured multi-GPU execution.
That weak association motivates probing candidates using live runs and updating predictions from those measurements. A prediction alone is not a dependable guide to which change will speed up a particular training setup.
- Compare tested workloads
Across four dense and mixture-of-experts workloads, InfraRSI achieved a geometric-mean speedup of 1.83x against each workload's reference configuration; RE-Bench reached 1.37x under the same task contracts. Auto-Research and settings-only OptiCo also used their native search rules.
The comparison supports InfraRSI's combined search on these workloads, while leaving open how much of the difference came from any individual mechanism.
- Reuse experience across hardware
A separate transfer test asks whether experience from other GPU types can help a new search. On one Qwen3.5-9B workload using 8 H100 GPUs, cross-hardware experience finished at 1.82x speedup, compared with 1.70x for both cold start and all eligible experience packs.
This supports filtering experience by hardware for that tested target, but the single-target comparison cannot establish the same effect elsewhere.
Deep
- Original paper authors
- InfraRSI Team, Yuchuan Tian, Haocheng Mei, Ye Yuan, Chao Xu
- Submitted to arXiv
- License
- arXiv non-exclusive