Research collection
#synthetic-data-curricula
1 paper tagged Synthetic data curricula. Each paper states what it shows and what it does not prove.
1 results
Synthetic training fails to beat text on fixed budgets
Tests of synthetic pre-pretraining find no fixed-budget language modeling advantage over natural-language-only training in the settings studied.
