Machine Learning
Research on Machine Learning: explanations, evidence, limitations and connections between papers.
37 results
A method separates data selection from support shifts
A construction-based method separates data selection from support shifts; retrieval evidence favors within-region selection in one data family.
A calibrated reward helps models follow rules and answer well
CaLIF raises added-rule compliance while keeping answer quality near base across five tested model configurations.
Reconstructing replies steadies jailbreak success reports
A reply reconstruction method steadies jailbreak evaluations, while human-labeled tests expose misses on incomplete answers.
Language models preserve useful past predictions with FSPO
Across five task accuracy averages, FSPO exceeded on-policy SDPO after five hours on two tested language models.
Direct updates usually beat tiled reuse for changing sparse inputs
Across tested moving inputs, direct column updates beat a tiled executor in most sessions; matrix structure shaped the exceptions.
LayerTrim compresses reasoning models while preserving their depth
LayerTrim trims selected model features and led tested pruning methods in accuracy across three reasoning models.
Near-one momentum improved validation accuracy in two tests
Bandit theory and tests on two language models find higher validation accuracy with longer-lasting gradient averages.
Choose evidence that matches the decision conditions
A model can look best because its test set scarcely represents conditions it will face. Across these studies, evidence supports a choice when it examines the relevant conditions at the level where that choice acts. Earlier clues can screen options within their tested scope, but the studies’ different tasks and outcomes cannot be combined into one performance ranking.
Why larger rollouts learn slower but fail on a similar schedule
A two-clock study finds larger rollouts need more responses to learn, while failure follows a separate pattern.
FlashMask-3 speeds structured attention across GPUs
FlashMask-3 coordinates structured attention masks across Hopper GPUs and raised throughput in one large-model training setup.
Interaction traces help judges assess software as people use it
Interaction traces improved tested website preference judgments, broken game checks, and repairs, with evidence limited to selected tasks.
Structured weights make Muon training cheaper at scale
In matched dense and MoE runs, Monarch–Muon had lower loss than Monarch–AdamW; at 6.89B, it cut step time and memory versus dense Muon.
A revised test narrows the compute gap for diffusion models
A compute-matched study finds exact scoring and encoder–decoder block diffusion narrow the measured language-model pretraining gap.
A token-wise update keeps stale-response training aimed at its target
Proximal KL has higher math response accuracy than GRPO in seven tested off-policy comparisons across two Qwen3 sizes.
Match each inference change to the operation under pressure
A long document can strain inference in several ways. Answering a query may require attention to many past items; a compressed cache must decide which items to retain; a recurrent model may repeatedly write a large state. These pressures call for different changes. The studies suggest first locating the operation under pressure. One reduces attention retrieval work, another selects stored cache entries, and a third reduces state memory traffic. Each result is useful within its own measured workload and outcome.
DuoRate lets MoE models store and read weights at different rates
DuoRate kept accuracy near a four-bit format while reading fewer bits per decode step on one MoE model.
On-policy distillation has no consistent edge in core tests
A controlled distillation study finds no consistent on-policy edge, with gains on harder arithmetic not reliably persisting after further training.
One language interface steers protein structure and binder design
Prompteina uses natural-language prompts to control protein structures, outperforming other systems on predicted interface confidence across 20 targets.
A visual-generation trainer makes reward learning faster and steadier
DiffusionART uses shared reward directions and restoration to stabilize image and video generator training in tested settings.
DualQuant quantizes language models without calibration data
DualQuant uses weight-derived scales to quantize language-model weights and activations without calibration data.
FORTE turns feature removal into a staged production workflow
A production framework tests feature removals jointly and online, enabling bounded efficiency gains in two recommender systems.
QUILT speeds long-context prefill by sharing query work
QUILT reuses overlapping key-value entries to reduce tested prefill latency with small measured score differences.
Static memory expands pretrained models without sparse pretraining
STAPLE adds static memory to pretrained models, raising tested scores while keeping decode throughput near dense models.
Tabular transformers can replace half their blocks with low classification loss
A label-free affine translator removes half the transformer blocks with minimal average classification AUROC loss.