Deep Autocorrelation Modeling for Time-Series Forecasting: Progress and Prospects
| Authors | Hao Wang et al. |
| Year | 2026 |
| Field | Statistics / ML |
| arXiv | 2603.19899 |
| Download | |
| Categories | stat.ML, cs.LG, stat.AP |
Abstract
Autocorrelation is a defining characteristic of time-series data, where each observation is statistically dependent on its predecessors. In the context of deep time-series forecasting, autocorrelation arises in both the input history and the label sequences, presenting two central research challenges: (1) designing neural architectures that model autocorrelation in history sequences, and (2) devising learning objectives that model autocorrelation in label sequences. Recent studies have made strides in tackling these challenges, but a systematic survey examining both aspects remains lacking. To bridge this gap, this paper provides a comprehensive review of deep time-series forecasting from the perspective of autocorrelation modeling. In contrast to existing surveys, this work makes two distinctive contributions. First, it proposes a novel taxonomy that encompasses recent literature on both model architectures and learning objectives -- whereas prior surveys neglect or inadequately discuss the latter aspect. Second, it offers a thorough analysis of the motivations, insights, and progression of the surveyed literature from a unified, autocorrelation-centric perspective, providing a holistic overview of the evolution of deep time-series forecasting. The full list of papers and resources is available at https://github.com/Master-PLC/Awesome-TSF-Papers.
Engineering Breakdown
Plain English
This paper surveys deep learning approaches for time-series forecasting with a focus on how neural networks model autocorrelation—the statistical dependency between observations in sequential data. The authors systematically review two complementary research challenges: designing architectures that capture autocorrelation patterns in historical input sequences, and creating learning objectives that model autocorrelation in prediction targets. Rather than treating these as separate problems, the survey positions autocorrelation modeling as the unifying lens for understanding modern deep forecasting methods, filling a gap where existing surveys address components independently but lack integrated perspective.
Core Technical Contribution
The core contribution is a comprehensive taxonomy and systematic framework for understanding deep time-series forecasting through the autocorrelation modeling perspective. Instead of organizing prior work by architecture type (RNNs, Transformers, etc.) or task variant, the authors structure the entire field around two dual challenges: architectural innovations that explicitly model historical autocorrelation, and objective functions designed to handle autocorrelation in label sequences. This dual-perspective framework reveals hidden connections between seemingly disparate methods and provides a principled way to evaluate whether new architectures or training approaches actually address fundamental autocorrelation challenges or merely achieve empirical gains through other mechanisms.
How It Works
The survey examines time-series forecasting as a sequence-to-sequence problem where given a historical window of observations (input history with inherent autocorrelation), the model must predict future values (output labels also exhibiting autocorrelation). For modeling input-side autocorrelation, the paper categorizes approaches: recurrent architectures that process sequences step-by-step preserving temporal dependencies, attention-based mechanisms that learn which past timesteps matter most, and hybrid architectures combining multiple pattern-capture methods. For modeling output-side autocorrelation, it reviews techniques including direct multi-step prediction (where earlier predictions influence later ones), iterative refinement objectives, and structured prediction losses that explicitly penalize violations of expected autocorrelation patterns. The framework systematically maps prior work into these categories, showing how different architectural choices and training objectives either implicitly or explicitly handle autocorrelation at each stage.
Production Impact
For engineers building forecasting systems, this survey provides a diagnostic framework to audit whether existing architectures genuinely handle autocorrelation or achieve accuracy through other mechanisms, which is critical for understanding when a model will generalize versus overfit to specific datasets. The dual-perspective approach suggests that optimal production systems need both sides addressed: using Transformer or dilated CNN architectures for input processing, paired with training objectives that explicitly constrain prediction consistency over time horizons. This has immediate implications for model selection—practitioners can now reason about whether their forecasting accuracy comes from modeling true temporal dependencies (which transfer across domains) or dataset artifacts (which won't generalize). The tradeoff is that explicitly modeling autocorrelation at both stages increases computational cost during training and may require careful regularization tuning, but the payoff is improved out-of-distribution generalization on different time-series domains.
Limitations and When Not to Use This
The paper is primarily a survey rather than a new method, so it does not propose novel architectures or training procedures; practitioners still face the challenge of deciding which reviewed approaches to implement given their specific constraints. The survey's framework assumes autocorrelation is the dominant structure in time-series data, but many real-world forecasting problems involve irregular patterns, distribution shifts, or exogenous variables where autocorrelation is secondary; the taxonomy may not cleanly apply to multivariate forecasting with complex interdependencies. The paper does not provide empirical comparisons quantifying which combinations of input-side and output-side autocorrelation modeling techniques work best for different data characteristics, leaving practitioners to navigate implementation tradeoffs experimentally. Additionally, the survey may not fully address scenarios where breaking autocorrelation assumptions (anomaly forecasting, regime changes, Black Swan events) is actually desirable, limiting applicability in high-risk domains.
Research Context
This work builds on and synthesizes decades of time-series analysis research (classical ARIMA/GARCH methods) and modern deep learning advances (RNNs, LSTMs, Transformers for sequence modeling), positioning autocorrelation as the conceptual bridge between classical statistics and modern neural methods. It advances recent work showing that simple statistical models sometimes outperform complex neural nets on standard benchmarks by proposing that the real differentiator is whether neural architectures genuinely model autocorrelation structure or merely memorize dataset patterns. The survey likely impacts evaluation of time-series forecasting, suggesting that benchmarks should isolate autocorrelation modeling capability rather than just minimizing overall prediction error. This opens research directions toward methods that explicitly disentangle autocorrelation from other temporal patterns, and toward theoretical analysis of what architectural choices are fundamentally necessary versus sufficient for different orders of autocorrelation.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::