Research collection
#momentum-scaling
Research tagged momentum-scaling: explanations, evidence, limitations and connections between papers.
1 results
Near-one momentum improved validation accuracy in two tests
Bandit theory and tests on two language models find higher validation accuracy with longer-lasting gradient averages.