Why delayed labels change forecasting adaptation results
In plain words
The paper asks whether software that adjusts a time-series forecaster can really help when the answers it learns from arrive late.
It introduces a test that releases each forecast’s target only when that target would actually be available, plus a simple reference adapter that corrects predictions as labels mature.
When tested on five data collections, some gains seen with early label access disappeared or became harm. On four collections outside ETTm1, the published methods’ biggest significant improvement over their frozen forecasts was only 0.5%.
The research
When a deployed forecaster receives the truth about a multi-step prediction several steps after making it, can an adapter still improve the next predictions, or do earlier tests reward access to information that was not yet available?
The paper asks this across different delays because real data pipelines can add lag beyond the time needed to observe the forecast window.
The paper enforces when forecast labels become usable and introduces a closed-form adapter, showing that many reported gains shrink or reverse under causal delays.
The authors replayed five multivariate forecasting streams with labels released at the minimum causal delay, longer delays, or a deliberately leaky early time.
They measured standardized mean squared error, the average squared forecast error after scaling with pretraining statistics, and compared four published adapters with their frozen checkpoints and two reference correctors.
The leaky protocol made bias-EMA look better than it was: causal error was 83% higher on ETTh2 with DLinear and 110% higher with PatchTST. Outside ETTm1, published methods improved over frozen models by at most 0.5%, supporting the claim that label timing changes the evaluation.
The reference adapter itself became significantly worse than the frozen model on ETTh2 when labels were delayed to 2H and 4H. The audit used one deep architecture, uneven checkpoint coverage, and a deliberately nonstandard split, so its absolute numbers are not directly comparable across papers.
Analysis
Deep
- License
- arXiv non-exclusive