Small GPU runs can guide large training searches
In plain words
Training a large language model across many GPUs involves choices about how to divide work and memory. Trying each choice on the full cluster can take hours and still fail.
Primus Projection estimates how a setup will perform at scale using smaller GPU runs or a CPU simulation. Its Tuning Agent uses those estimates to search for promising settings. In the reported AMD GPU sweeps, projected throughput stayed within about 10% of measured throughput.
The research
When a large training run can take hours and still fail for lack of memory, how can teams choose settings without testing every configuration on the full cluster? The paper asks whether measurements from a smaller run can help identify memory-legal, high-throughput settings at the target scale.
Primus Projection estimates memory and target-scale throughput from smaller GPU runs, so a tuning agent can screen and rank training configurations.
To test whether smaller runs can guide choices at larger scales, the authors compared Primus projections with hardware measurements.
They benchmarked on 1, 2, 4 or 8 GPUs on one host, then restored the target analytically; across the tested AMD Llama and Mixtral sweeps, projected throughput differed from measurements by no more than about 10%.
Because the tuning agent selects configurations by projected scores, the authors also compared rankings with measured rankings. Rankings matched exactly for Mixtral and Qwen3 and closely for GPT-OSS-120B; the agent chose the measured-best candidate in each sweep. This supports using projections to guide search in the tested cases.
The evidence supports projection-guided search on the tested models and AMD systems. It does not establish accuracy on other hardware, model network contention, or show whether FP8 recipes converge.
Analysis
Deep
- Original paper authors
- Anshu Raina, Peyman Razaghi, Yuankai Chen, Cheng Yao, Yao Fu
- Submitted to arXiv
- License
- arXiv non-exclusive