Machine Learning
Research on Machine Learning: explanations, evidence, limitations and connections between papers.
29 results
Why larger rollouts learn slower but fail on a similar schedule
A two-clock study finds larger rollouts need more responses to learn, while failure follows a separate pattern.
FlashMask-3 speeds structured attention across GPUs
FlashMask-3 coordinates structured attention masks across Hopper GPUs and raised throughput in one large-model training setup.
Interaction traces help judges assess software as people use it
Interaction traces improved tested website preference judgments, broken game checks, and repairs, with evidence limited to selected tasks.
Structured weights make Muon training cheaper at scale
In matched dense and MoE runs, Monarch–Muon had lower loss than Monarch–AdamW; at 6.89B, it cut step time and memory versus dense Muon.
A revised test narrows the compute gap for diffusion models
A compute-matched study finds exact scoring and encoder–decoder block diffusion narrow the measured language-model pretraining gap.
A token-wise update keeps stale-response training aimed at its target
Proximal KL has higher math response accuracy than GRPO in seven tested off-policy comparisons across two Qwen3 sizes.
What to Read, Keep, and Write During Model Inference
A model serving long documents may spend work finding past information, run short of space for stored document entries, or repeatedly move a recurrent state while generating an answer. Those pressures call for different changes. The studies suggest a practical first question: which operation is costly in the intended workload? One changes attention retrieval, one chooses what a compressed cache retains, and one writes recurrent state less often. Their reported outcomes belong to different tasks and measures, so each is useful within its own tested conditions.
DuoRate lets MoE models store and read weights at different rates
DuoRate kept accuracy near a four-bit format while reading fewer bits per decode step on one MoE model.
On-policy distillation has no consistent edge in core tests
A controlled distillation study finds no consistent on-policy edge, with gains on harder arithmetic not reliably persisting after further training.
One language interface steers protein structure and binder design
Prompteina uses natural-language prompts to control protein structures, outperforming other systems on predicted interface confidence across 20 targets.
A visual-generation trainer makes reward learning faster and steadier
DiffusionART uses shared reward directions and restoration to stabilize image and video generator training in tested settings.
DualQuant quantizes language models without calibration data
DualQuant uses weight-derived scales to quantize language-model weights and activations without calibration data.
FORTE turns feature removal into a staged production workflow
A production framework tests feature removals jointly and online, enabling bounded efficiency gains in two recommender systems.
QUILT speeds long-context prefill by sharing query work
QUILT reuses overlapping key-value entries to reduce tested prefill latency with small measured score differences.
Static memory expands pretrained models without sparse pretraining
STAPLE adds static memory to pretrained models, raising tested scores while keeping decode throughput near dense models.
Tabular transformers can replace half their blocks with low classification loss
A label-free affine translator removes half the transformer blocks with minimal average classification AUROC loss.
Why delayed labels change forecasting adaptation results
A delay-aware audit finds forecasting gains often shrink or reverse when labels arrive after forecasts mature.
Diffusion models can reveal memorized images before copies
A diffusion-model method probes hidden training-image memorization, with evidence bounded to tested datasets and one released model.
Antenna agreement helps recover breathing under motion interference
A spatial adaptation method improves breathing-waveform recovery under simulated motion interference and lowers failures after real-data calibration.
A lattice method predicts better decoding schedules before generation
A lattice method estimates lost position dependence to rank decoding schedules before generation, with tested image evidence.
Reaching the Goal, Responding in Time, and Staying Safe
A robot can complete a task and still respond too late to a moving object. It can also reach a destination after a collision. Saying that it “succeeded” leaves out what was measured and what happened while it decided. The selected studies offer a bounded answer: adapting computation was checked against task completion, rewarding varied successful behavior was checked under shifted simulation conditions, and a moving-world test separately counted safe arrivals. Each claim belongs to its own test.
What Else Changes When You Fix a Model?
A model can learn from a new corpus while losing old behavior, receive privacy-protecting noise while losing task accuracy, or refuse a harmful prompt while also refusing a harmless neighbor. Success at the intended target leaves a nearby question unanswered. The studies suggest a bounded way to investigate: locate what changes, choose a comparator, and inspect the closest relevant boundary. That boundary is different for training updates, privacy calibration, and prompt repair. Neither a plausible mechanism nor a reassuring broad score settles the other studies’ questions.
Branching diffusion cuts the work of long text generation
Branching Latent Diffusion cuts computation for long text generation, while quality and single-sequence speed remain bounded.
SoftmaxLift reduces key retrieval for long-context attention
SoftmaxLift augments keys and queries to estimate attention with one exact search index and O(1/ε²) retrieved keys.