Research collection
#matrix-aware-optimization
Research tagged matrix-aware-optimization: explanations, evidence, limitations and connections between papers.
1 results
Structured weights make Muon training cheaper at scale
In matched dense and MoE runs, Monarch–Muon had lower loss than Monarch–AdamW; at 6.89B, it cut step time and memory versus dense Muon.