Research collection
#per-microbatch-memory
Research tagged per-microbatch-memory: explanations, evidence, limitations and connections between papers.
1 results
A training system adapts memory to rare expert-load spikes
A system adjusts activation memory during expert-routing spikes and maintains steady computation use in one 512-GPU training run.