What-if forecasts can mistake association for causal effect
Forecasting Associations, Not Interventions: A Preregistered Causal Audit of Covariate-Aware Time Series Foundation Models · arXiv:2610.31869
In plain words
Time-series models can accept a possible future path for a control, such as an actuator setting, and show how their forecast changes. But these models learn from observations, where the control may be linked to other causes of the outcome. A changed forecast may show that link rather than the effect of actually changing the control.
The paper introduces an audit that compares answers from observational and randomized data. On synthetic tests, four models showed confounding at least as large as a regression reference in key settings. Training one model further reduced the gap, but did not remove it.
The research
When a forecasting model accepts a possible future path for a control, people may read the change in its forecast as the effect of changing that control. But if the model learned from observations, that control may be associated with other causes of the outcome. Does its answer reflect an intervention, or only an association?
The paper introduces an audit that compares forecast effects in observational and randomized contexts to measure how much association enters a what-if answer.
The authors compare each model’s estimated effect in observational contexts with its estimate in matched randomized contexts, then check how the resulting gap compares with a least-squares reference. On a synthetic grid, four models tracked or exceeded that reference when the confounder was hidden or past-only.
The pattern indicates that their what-if answers can carry observational association into the forecast change.
A matched-pair test found that when future confounder information made different causal effects identifiable, least squares recovered the full difference, while Chronos-2 and TiRex-2 recovered only part.
Training Chronos-2 further reduced its synthetic confounding gap, with forecasting error within the stated limit on an hourly test.
The theory is proved for linear-Gaussian models, while nonlinear and physical-system findings are empirical and cover limited configurations. Post-training reduced the gap on synthetic data but did not remove all bias, and its response scale varied between physical systems.
What the evidence shows
- A what-if question
The paper introduces a theory-anchored causal audit, a method for checking whether a covariate-aware time-series model’s forecast change under a supplied future control path reflects an intervention’s causal effect or an observational association.
The question matters because these models learn from observed data, where a control can move with another cause of the outcome. Their forecast difference alone may therefore mix the control’s association with its effect.
- A test for confounding
The audit compares a model’s estimated effect in observational contexts with its estimate in randomized contexts, keeping the structural model, query and confounder visibility matched. This difference, called the confounding gap, measures how much association enters the what-if answer without requiring the true effect.
The paper’s theory explains why observational prediction can produce this association and when context information can block it.
- A reference and result
The authors first checked the audit against an explicit least-squares forecaster: across the reported synthetic visibility settings, its measured gap matched the paper’s closed-form prediction. They then evaluated four forecasting models on a preregistered synthetic grid.
All tracked or exceeded the least-squares confounding reference when the confounder was hidden or past-only; three showed larger gaps, while TabPFN-TS followed the reference.
- Can models use the clues?
A matched-pair test distinguished missing information from failure to use it. When paired synthetic models had identical observed relationships but different effects, the effects could not be distinguished with hidden or past-only confounder information.
When the confounder’s future made them identifiable, least squares recovered the full difference, while Chronos-2 recovered 75% and TiRex-2 34%.
- A partial repair
Training Chronos-2 further on interventional series with replay from a pretraining data family reduced its confounding gap on synthetic tests. The forecasting error increase on a separate hourly forecasting test remained within the paper’s 5% limit.
Physical-system tests, however, showed that the repaired model still under-responded in one system and overshot in another, so the repair did not establish a uniform response scale.
Deep
- Original paper authors
- Zong-Han Bai
- Submitted to arXiv
- License
- CC BY 4.0