Time-Series Features

The Forecast That Worked Perfectly in Training

The demand forecasting model looked extraordinary during development. Mean Absolute Percentage Error (MAPE) of 4.2% on the test set - well below the business target of 8%. The team celebrated. The model was deployed to drive inventory planning for a chain of 340 retail stores.

In the first week of production, actual demand deviated from forecasts by an average of 23%. In the second week, 19%. Inventory decisions made on these forecasts led to overstocking in some categories and stockouts in others. The business impact was material. A post-mortem was called.

The data science team reviewed the model. The architecture was sound. The hyperparameters were tuned correctly. The training data covered three years of historical demand. So why was the test set error 4% but the production error 23%?

The answer was in how the test set had been constructed. A junior data scientist had split the data randomly - 80% of records to training, 20% to test, shuffled uniformly. This meant the training set contained demand records from 2024 and the test set contained demand records from 2021. The model was being evaluated on the past using features computed from the future.

The most damaging feature was rolling_28d_avg_demand - the 28-day rolling average of demand preceding each record. When computed without temporal awareness, a record from January 15 would have its rolling average computed using data from January 10 to February 7 - data that did not exist in January. The feature was looking into the future. The model achieved a 4% MAPE by essentially memorizing the data.

This is temporal leakage. It is the most common mistake in time-series machine learning and the one with the most devastating production consequences.

---

:::tip 🎮 Interactive Playground Visualize this concept: Try the Feature Engineering demo on the EngineersOfAI Playground - no code required. :::

Why This Exists: The Temporal Structure Problem

Standard machine learning assumes that training examples are independent and identically distributed (i.i.d.). Time series data violates this assumption: observations are ordered in time, and the relationship between observations at adjacent times is the primary source of predictive signal.

When you ignore temporal structure, you make errors that seem like model quality issues but are actually data engineering issues:

• Temporal leakage: Features computed using future data appear highly predictive in training but are impossible to compute in production
• Random splits: Shuffling time-series data before train/test split puts future data in the training set and past data in the test set - the model learns the test set from the training set
• Stale features: Features computed on the wrong time window (e.g., "yesterday's demand" computed using today's data) produce optimistic offline results and degraded production performance

Time-series feature engineering is the discipline of extracting predictive signal from temporal data while respecting the causal structure of time: you can only use information from the past to predict the future.

---

Historical Context

Time-series feature engineering for machine learning (as opposed to classical statistical forecasting) grew rapidly with the Kaggle M4 and M5 competitions. The M5 competition (2020), using Walmart sales data, demonstrated that gradient boosting models with carefully engineered lag and rolling features could outperform classical statistical forecasting approaches like SARIMA and Holt-Winters.

The competition winners consistently used similar feature sets: multiple lag windows, multiple rolling aggregation windows, calendar features, and store-level hierarchy embeddings. This standardized a vocabulary of temporal features that has since become the industry norm.

The temporal leakage problem was formally described in the context of financial ML by Lopez de Prado (2018) in "Advances in Financial Machine Learning," which introduced the concept of embargo periods and purged cross-validation specifically to prevent future data from contaminating training samples.

---

Core Concepts

Lag Features

A lag feature is the value of a time series at a previous time step. If you want to predict demand at time $t$, the demand at time $t-1$, $t-7$, $t-14$, and $t-28$ are natural lag features. They capture autocorrelation - the tendency of a time series to resemble its own past.

The lag period should be chosen based on the domain. For weekly data with strong weekly seasonality, lags of 1, 4, 8, and 52 weeks capture the relevant autocorrelation structure. For daily retail data, lags of 1, 7, 14, 28, and 365 days are natural.

The leakage rule: When computing lag features for a prediction at time $t$, you may only use data from time $t - \text{lag}$ or earlier. If your lag feature requires data from after $t$, it will not be available at serving time.

import pandas as pd
import numpy as np
from typing import List

def add_lag_features(
    df: pd.DataFrame,
    target_col: str,
    entity_col: str,       # e.g., "store_id" - compute lags within each entity
    time_col: str,
    lag_periods: List[int]
) -> pd.DataFrame:
    """
    Add lag features while respecting temporal ordering within each entity.
    """
    df = df.sort_values([entity_col, time_col]).copy()

    for lag in lag_periods:
        feature_name = f"{target_col}_lag_{lag}"
        df[feature_name] = df.groupby(entity_col)[target_col].shift(lag)
        # shift(lag) returns NaN for the first `lag` rows of each group
        # This is correct - we don't have data for those periods

    return df


# Example: retail demand with multiple lag periods
demand_df = pd.DataFrame({
    "store_id": ["s1"] * 100 + ["s2"] * 100,
    "date": list(pd.date_range("2024-01-01", periods=100)) * 2,
    "demand": np.random.poisson(100, 200).tolist()
})

lag_periods = [1, 7, 14, 28, 56, 91, 182, 365]
demand_df = add_lag_features(
    df=demand_df,
    target_col="demand",
    entity_col="store_id",
    time_col="date",
    lag_periods=lag_periods
)

print(f"Features added: {[f for f in demand_df.columns if 'lag' in f]}")
print(f"NaN rows (expected for early lags): {demand_df['demand_lag_365'].isnull().sum()}")

Rolling Statistics

Rolling statistics summarize the behavior of a time series over a moving window. They capture trend, volatility, and seasonality that lag features alone don't convey.

Common rolling statistics:

• Mean: trend, smoothed level
• Standard deviation: volatility
• Min/Max: range, extreme events
• Skewness: asymmetry of distribution over the window
• Median: robust trend (less sensitive to outliers than mean)

The choice of window size should reflect the temporal scale of the patterns you're trying to capture. A 7-day window captures weekly patterns; a 28-day window captures monthly patterns.

def add_rolling_features(
    df: pd.DataFrame,
    target_col: str,
    entity_col: str,
    time_col: str,
    window_sizes: List[int],
    min_periods_fraction: float = 0.5
) -> pd.DataFrame:
    """
    Add rolling statistics, computed within each entity group.
    min_periods_fraction: minimum fraction of window that must be non-null.
    """
    df = df.sort_values([entity_col, time_col]).copy()

    for window in window_sizes:
        min_p = max(1, int(window * min_periods_fraction))

        grp = df.groupby(entity_col)[target_col]

        # Shift by 1 before rolling to exclude the current period
        # This ensures no leakage: rolling features use only past data
        shifted = grp.shift(1)
        rolling = shifted.groupby(df[entity_col]).rolling(
            window=window, min_periods=min_p
        )

        df[f"{target_col}_roll_{window}_mean"]  = rolling.mean().values
        df[f"{target_col}_roll_{window}_std"]   = rolling.std().values
        df[f"{target_col}_roll_{window}_min"]   = rolling.min().values
        df[f"{target_col}_roll_{window}_max"]   = rolling.max().values
        df[f"{target_col}_roll_{window}_median"] = rolling.median().values

    return df

# Note: shift(1) before rolling is critical - without it, the current
# observation is included in its own rolling window. This is leakage.
demand_df = add_rolling_features(
    df=demand_df,
    target_col="demand",
    entity_col="store_id",
    time_col="date",
    window_sizes=[7, 14, 28, 56, 91]
)

Expanding Window Features

An expanding window starts at the beginning of the series and grows with each new observation. Unlike a fixed rolling window, it summarizes all available history.

Expanding features are useful for capturing long-run trends, cumulative effects, and entity-level baselines.

def add_expanding_features(
    df: pd.DataFrame,
    target_col: str,
    entity_col: str,
    time_col: str
) -> pd.DataFrame:
    df = df.sort_values([entity_col, time_col]).copy()

    grp = df.groupby(entity_col)[target_col]

    # Shift by 1 to exclude current observation
    shifted = grp.shift(1)

    expanding = shifted.groupby(df[entity_col]).expanding(min_periods=7)

    df[f"{target_col}_expanding_mean"] = expanding.mean().values
    df[f"{target_col}_expanding_std"]  = expanding.std().values
    df[f"{target_col}_expanding_max"]  = expanding.max().values

    # Historical percentile rank - where does current demand rank historically?
    df[f"{target_col}_historical_rank"] = (
        grp.expanding().rank(pct=True).values
    )

    return df

Fourier Features for Seasonality

Seasonal patterns repeat at fixed intervals: daily (24-hour), weekly (7-day), annual (365-day). Fourier features encode these cycles as sine and cosine pairs, allowing linear and tree-based models to capture smooth seasonal patterns without needing explicit calendar lookups.

For a period $P$, the $k$-th Fourier pair is:

$\sin\left(\frac{2\pi k t}{P}\right), \quad \cos\left(\frac{2\pi k t}{P}\right)$

where $t$ is the time index and $k = 1, 2, ..., K$ controls the number of harmonics. Higher $k$ captures finer-grained seasonal patterns.

def add_fourier_features(
    df: pd.DataFrame,
    time_col: str,
    periods: dict,    # e.g., {"weekly": 7, "annual": 365}
    n_harmonics: int = 3
) -> pd.DataFrame:
    """
    Add Fourier features for multiple seasonal periods.
    periods: dict mapping period name to period length (in days)
    n_harmonics: number of sine/cosine pairs per period
    """
    df = df.copy()

    # Convert date to numeric time index (days since origin)
    t = (df[time_col] - df[time_col].min()).dt.days.values

    for period_name, period_length in periods.items():
        for k in range(1, n_harmonics + 1):
            angle = 2 * np.pi * k * t / period_length
            df[f"fourier_{period_name}_sin_{k}"] = np.sin(angle)
            df[f"fourier_{period_name}_cos_{k}"] = np.cos(angle)

    return df


demand_df["date"] = pd.to_datetime(demand_df["date"])
demand_df = add_fourier_features(
    df=demand_df,
    time_col="date",
    periods={"weekly": 7, "annual": 365},
    n_harmonics=3
)
# Creates: fourier_weekly_sin_1, fourier_weekly_cos_1, ..., fourier_annual_sin_3, ...

Calendar and Holiday Features

Holidays, promotions, and day-of-week effects are strong drivers of demand that Fourier features alone don't capture (since holidays are irregularly spaced). Encoding these explicitly gives the model the ability to learn promotion effects and holiday lift.

def add_calendar_features(df: pd.DataFrame, time_col: str) -> pd.DataFrame:
    df = df.copy()
    dates = pd.to_datetime(df[time_col])

    # Basic calendar features
    df["day_of_week"]  = dates.dt.dayofweek          # 0=Monday, 6=Sunday
    df["day_of_month"] = dates.dt.day
    df["week_of_year"] = dates.dt.isocalendar().week.astype(int)
    df["month"]        = dates.dt.month
    df["quarter"]      = dates.dt.quarter
    df["year"]         = dates.dt.year
    df["is_weekend"]   = (dates.dt.dayofweek >= 5).astype(int)
    df["is_month_start"] = dates.dt.is_month_start.astype(int)
    df["is_month_end"]   = dates.dt.is_month_end.astype(int)

    # Days since/until known events (e.g., Black Friday - roughly Nov 25)
    # Use actual dates from a holiday calendar in production
    black_friday_2024 = pd.Timestamp("2024-11-29")
    df["days_to_black_friday"] = (black_friday_2024 - dates).dt.days.clip(lower=0)
    df["days_since_black_friday"] = (dates - black_friday_2024).dt.days.clip(lower=0)

    # Cyclical encoding for periodic calendar features
    # Prevents the discontinuity between Dec (12) and Jan (1)
    df["month_sin"] = np.sin(2 * np.pi * df["month"] / 12)
    df["month_cos"] = np.cos(2 * np.pi * df["month"] / 12)
    df["dow_sin"]   = np.sin(2 * np.pi * df["day_of_week"] / 7)
    df["dow_cos"]   = np.cos(2 * np.pi * df["day_of_week"] / 7)

    return df

Time-Since Features

Time-since features capture the recency of events. "Days since last purchase," "days since last complaint," "days since price changed" encode information about the state of an entity's relationship with a product or service.

def add_time_since_features(
    df: pd.DataFrame,
    event_df: pd.DataFrame,  # events with entity_id and event_timestamp
    entity_col: str,
    time_col: str,
    event_col: str,          # event type column
    event_types: List[str]
) -> pd.DataFrame:
    """
    For each row in df, compute days since each event type last occurred
    for that entity, as of the row's timestamp.
    """
    df = df.sort_values([entity_col, time_col]).copy()

    for event_type in event_types:
        events_of_type = event_df[event_df[event_col] == event_type]

        merged = df.merge(
            events_of_type[[entity_col, "event_timestamp"]],
            on=entity_col,
            how="left"
        )

        # Only keep events that happened BEFORE the current row's timestamp
        merged = merged[merged["event_timestamp"] <= merged[time_col]]

        # Take the most recent event before each row
        last_event = merged.groupby([entity_col, time_col])["event_timestamp"].max().reset_index()
        last_event.columns = [entity_col, time_col, f"last_{event_type}_timestamp"]

        df = df.merge(last_event, on=[entity_col, time_col], how="left")
        df[f"days_since_{event_type}"] = (
            pd.to_datetime(df[time_col]) -
            pd.to_datetime(df[f"last_{event_type}_timestamp"])
        ).dt.days

    return df

Temporal Cross-Validation

The standard K-fold cross-validation shuffles data randomly before splitting - which is catastrophically wrong for time series. Observations from the future end up in the training fold, leaking information backward.

Time-series cross-validation uses walk-forward validation: the training window grows forward in time, and the validation window always follows the training window, never overlapping.

from sklearn.model_selection import TimeSeriesSplit
import numpy as np

def temporal_cross_validation(
    X: pd.DataFrame,
    y: pd.Series,
    model,
    n_splits: int = 5,
    gap: int = 7    # days gap between train and validation - prevents leakage from slow-updating features
):
    """
    Walk-forward cross-validation with a gap between train and validation.
    The gap prevents leakage when features have a slow update frequency.
    """
    tscv = TimeSeriesSplit(n_splits=n_splits, gap=gap)
    scores = []

    for fold_n, (train_idx, val_idx) in enumerate(tscv.split(X)):
        X_train, X_val = X.iloc[train_idx], X.iloc[val_idx]
        y_train, y_val = y.iloc[train_idx], y.iloc[val_idx]

        model.fit(X_train, y_train)
        preds = model.predict(X_val)

        mape = np.mean(np.abs((y_val - preds) / (y_val + 1e-9))) * 100
        scores.append(mape)
        print(f"Fold {fold_n+1}: MAPE = {mape:.2f}%")

    print(f"\nMean MAPE: {np.mean(scores):.2f}% ± {np.std(scores):.2f}%")
    return scores

---

Production Engineering Notes

Feature freshness: Lag and rolling features are only correct as of the time they are computed. A "7-day rolling average" computed yesterday and cached in a feature store is actually an 8-day rolling average today. For features with tight latency requirements, use a streaming feature pipeline (Flink/Spark Streaming) that recomputes rolling features as new data arrives.

Cold start: A new store or entity has no historical data. Lag features will be null for the first N periods, where N equals the lag period. Impute with global or cluster-level averages during the cold start period. Build explicit "entity age" features to let the model learn cold-start behavior.

Scalability: Computing rolling features over 365 days for millions of entities requires O(entities × time_steps × windows) work. Use Spark window functions for distributed computation, and compute incrementally (update only the latest window, not the full history).

---

Common Mistakes

:::danger Random train/test split for time series Splitting time series data randomly puts future observations in the training set and past observations in the test set. The model learns to predict the past from the future - an impossible task in production. Always split on a time boundary: all data before date D is training, all data after is test. :::

:::danger Rolling features without shift(1) Computing df["demand"].rolling(7).mean() without first applying .shift(1) includes the current observation in the rolling average used to predict that observation. This is leakage: you're using what you're trying to predict as an input to the prediction. Always shift by 1 before computing rolling features. :::

:::warning Not using a gap in temporal cross-validation If rolling features use a 7-day window, and your train/val boundary has no gap, then the rolling feature for the first day of validation was computed using training data. Use a gap equal to the longest rolling window to ensure strict temporal separation. :::

:::tip Use multiple lag scales Do not pick a single lag period. Use a range of lags at multiple scales: yesterday (1), last week (7), last month (28), last quarter (91). Different patterns manifest at different temporal scales. The model will learn which lags are predictive through feature importance and selection. :::

---

Interview Q&A

Q: What is temporal leakage and how do you prevent it?

A: Temporal leakage occurs when information from the future is used to predict the past during model training - typically through features computed using data that would not have been available at the time of prediction. The most common sources are: rolling features computed over windows that extend into the future relative to the label, random train/test splits that put future observations in the training set, and lag features that don't correctly shift the time axis. Prevention: always split on a time boundary, always apply shift(1) before rolling computations, use explicit walk-forward cross-validation with a gap, and validate that each feature's "knowledge cutoff" is before the label timestamp for every training example.

Q: What are Fourier features and when do you use them for demand forecasting?

A: Fourier features are sine/cosine pairs at specific frequencies that encode seasonal patterns. A weekly Fourier feature completes one cycle every 7 days; an annual Fourier feature completes one cycle every 365 days. They allow linear models and gradient boosting to capture smooth, periodic patterns without explicit calendar lookups. Use them when you have strong regular seasonality (retail demand, energy consumption, web traffic) and want the model to interpolate smoothly across the seasonal cycle rather than learning abrupt day-of-week transitions. The number of harmonics (K) controls how much detail the model captures - higher K captures finer seasonal structure at the risk of overfitting.

Q: How do you handle cold start for new entities in a time-series model?

A: New entities have no lag or rolling features - they're all null. Several strategies: (1) Global imputation: fill with the global mean or median across all entities. Simple, often reasonable for the first few periods. (2) Cluster-level imputation: group similar entities and impute with cluster-level statistics. Requires a clustering or entity similarity model. (3) Explicit cold-start features: add a binary is_new_entity flag and a entity_age_days feature. Let the model learn that young entities behave differently. (4) Separate model for new entities: train a simpler model with fewer features for cold-start periods, switch to the full model after 30+ days of history. In practice, a combination of cluster imputation and an explicit entity-age feature works well.

Q: What is walk-forward cross-validation and why is it necessary for time series?

A: Walk-forward (or rolling-origin) cross-validation is a temporal validation scheme where the training set grows forward in time and the validation set always follows the training set without overlap. In the first fold, you might train on months 1–6 and validate on month 7. In the second fold, train on months 1–7, validate on month 8. And so on. It's necessary because standard K-fold shuffles data randomly, which for time series means training on the future and validating on the past - a task that is trivial for the model but impossible in production. Walk-forward cross-validation simulates the actual deployment scenario: you always train on the past and predict the future.

Q: How do you scale rolling feature computation to millions of entities and years of data?

A: Three approaches. First, use Spark window functions for distributed computation. Spark's Window.partitionBy(entity_id).orderBy(timestamp).rangeBetween(-7*86400, -1) computes per-entity rolling windows in parallel across the cluster. Second, use incremental computation: given yesterday's rolling features, today's update only needs the latest batch of data plus the existing features - you don't recompute from scratch. This reduces computation by 90%+ for daily jobs. Third, precompute and cache in a feature store. Rolling features for all entities are computed once by the batch pipeline and cached in the offline store. Training jobs read the precomputed features instead of recomputing them. The expensive computation happens once; it's read many times.