Anomaly Detection in Sequences
| Reading time | Interview relevance | Target roles |
|---|---|---|
| ~45 minutes | Very High | MLE, AI Engineer, MLOps, Data Scientist |
The Production Incident That Changed Everything
It was 2:47 AM on a Tuesday when the on-call engineer at a mid-size European payment processor got paged. Not because an alert fired - but because a senior fraud analyst had manually spotted something wrong while reviewing the morning reconciliation reports. Over the previous six hours, the system had processed 4.2 million transactions. Of those, 847 were fraudulent. The existing anomaly detection system had caught 11 of them.
The fraud was not obvious. No single transaction was wildly out of range. There were no million-dollar transfers, no purchases from sanctioned countries. Instead, there was a pattern: accounts that had been dormant for 60 to 90 days suddenly initiated a sequence of small purchases - a 2.30 coffee there - spread across 8 to 12 hours. Each transaction, looked at in isolation, was completely normal. The account had a history of small purchases. The merchant categories matched the user's past behavior. The amounts were within historical ranges. Every point-in-time check passed green.
What the system was missing was the sequence. The dormancy followed by a sudden burst of activity, all within a narrow time window, at merchants the user had never visited, with a cadence that matched known card-testing patterns - this was invisible to a detector that treated each transaction as an independent event. The system was good at catching the obvious: a 200. It was completely blind to the temporal structure that made these 847 transactions a coordinated attack.
The engineering team spent the next three months rebuilding their detection stack. They moved from a purely feature-based point classifier to a system that explicitly modeled user transaction sequences. They trained LSTM autoencoders on historical transaction sequences, learning what "normal" activity looked like for each user segment. They added CUSUM detectors on rolling aggregates. They implemented dynamic thresholds that adapted to daily and weekly seasonality. By the end, their detection rate on sequence-based fraud went from 1.3% to 61% - with a false positive rate that actually dropped, because they were no longer flagging unusual-but-legitimate single transactions.
This lesson teaches you how to build that kind of system. The conceptual foundation, the mathematical tools, the code, and the production engineering discipline that separates a research prototype from something you can run at 4.2 million transactions per day.
Why This Exists - Why Standard Anomaly Detection Fails on Sequences
Before explaining what sequence anomaly detection is, you need to understand the specific ways that standard approaches break down. This is not a subtle failure - it is a fundamental architectural mismatch.
The i.i.d. Assumption Is a Lie
Classical anomaly detection - Isolation Forest, One-Class SVM, statistical z-scores - assumes that observations are independent and identically distributed. Each data point is evaluated based on its own feature values, with no reference to what came before or after it. This assumption is explicitly stated in the Isolation Forest paper (Liu et al., 2008): "We assume the data has i.i.d. distribution." For tabular data with no temporal structure, this is often fine. For sequences, it is catastrophically wrong.
In a sequence, the meaning of a value depends on its context. A heart rate of 140 BPM is:
- Normal if you just started running
- Alarming if you have been sitting quietly for an hour
- Expected if it follows a reading of 135 BPM ten seconds ago
- Suspicious if it follows a reading of 60 BPM with no intermediate steps
The number 140 carries almost no information without the sequence around it. The anomaly, if there is one, lives in the transition - in the relationship between past and present.
Three Kinds of Failures
Failure mode 1: False negatives on contextual anomalies. A server's CPU usage spikes to 95% every weekday at 9 AM when employees log in. A point detector trained on raw CPU values will flag this as an anomaly (it is statistically high). Meanwhile, a CPU reading of 35% on a Sunday at 2 AM on a server that normally sits at 0% during off-hours gets no flag. The Sunday reading is the actual anomaly - unexpected activity during a maintenance window - but its absolute value is unremarkable.
Failure mode 2: False negatives on collective anomalies. A network intrusion might consist of 500 individual packets, each of which looks completely normal in isolation (valid source IP, standard port, reasonable payload size). The anomaly is the collection - the specific sequence and timing of those packets forms a port scan pattern that no single packet reveals.
Failure mode 3: High false positive rates from ignoring temporal correlation. If a time series has strong autocorrelation (which most do), successive observations are not independent. A standard detector that does not account for this will fire constantly during legitimate trend changes, seasonality peaks, or any time the series evolves in a correlated way.
Historical Context - Where This Field Came From
Statistical Process Control, 1920s–1950s
The earliest formalization of anomaly detection in sequences comes from manufacturing quality control. Walter Shewhart at Bell Labs developed control charts in 1924 - the idea that a process producing measurements over time has a stable distribution, and points outside that distribution (typically beyond 3 standard deviations) signal that the process has changed. This gave us the vocabulary: "in control" vs "out of control," "special cause variation" vs "common cause variation."
The CUSUM (Cumulative Sum) algorithm, developed by E.S. Page in 1954, was a direct improvement on Shewhart charts. Rather than looking at each point independently, CUSUM accumulates deviations from the target. Small persistent deviations that would never cross a single-point threshold get caught because the cumulative sum eventually crosses its limit. This was the first algorithm explicitly designed to detect sequences of anomalies - the first algorithm to care about temporal structure.
Machine Learning Era, 2000s–2010s
Isolation Forest (Liu, Ting and Zhou, 2008) brought ensemble tree methods to anomaly detection. The core insight: anomalies are "few and different" - they are isolated easily by random partitioning because they live in sparse regions of feature space. Isolation Forest became the default baseline for tabular anomaly detection because it is fast, requires no label data, and scales well. Its limitation: it treats each row independently.
The deep learning push came with LSTM autoencoders around 2015 (Malhotra et al., "Long Short Term Memory Networks for Anomaly Detection in Time Series," 2015). The idea: train an LSTM encoder-decoder on normal sequences. At inference time, measure reconstruction error. Sequences the model has not "seen" during training - anomalies - will have high reconstruction error. This was the first widely practical approach that explicitly modeled temporal dependencies for anomaly detection.
Transformer Era, 2022
TranAD (Tuli et al., "TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data," 2022) applied transformer self-attention to sequence anomaly detection. Rather than relying on recurrent state, TranAD uses adversarial training with a transformer backbone to amplify reconstruction errors for anomalous sequences. It set new state-of-the-art benchmarks on MSL, SMAP, and SMD datasets. The key contribution: transformers can attend to non-local dependencies in the sequence, catching anomalies that depend on events far back in time - something LSTMs struggle with.
Taxonomy of Anomalies in Sequences
Before building a detector, you must decide what kind of anomaly you are trying to catch. The three categories are not interchangeable - different architectures are suited to different types.
Point Anomalies
A single observation is anomalous with respect to the rest of the data, regardless of context.
Example: A temperature sensor reads 450°C for one second in a server room that normally sits at 22°C. Even without any sequence context, this single reading is impossible.
Why sequence models still help here: Even for point anomalies, the local context improves detection. A value of 25°C is not anomalous globally, but if the last 60 readings were 22.1, 22.0, 22.2, 21.9, 22.1, and then suddenly 25.0 appears, the jump is the signal, not the absolute value.
Detectors that work: Rolling z-score, CUSUM, Isolation Forest on sliding windows.
Contextual Anomalies
An observation is anomalous given its context, but would be normal in a different context.
Example: A user transfers $50,000 in a single transaction. Normal if it is a payroll transfer that happens every two weeks on Friday. Anomalous if it is the first large transfer in an account's history, happening at 3 AM on a Sunday.
Example: A web server receives 10,000 requests per minute. Normal during a product launch. Anomalous at 4 AM on a holiday.
Why standard detectors fail here: A point detector trained on the global distribution of transfer amounts will either flag the payroll transfer every two weeks (high false positive rate) or will have a threshold high enough that it misses unusual large transfers (false negatives). The only way to detect contextual anomalies reliably is to model the expected value given the context.
Detectors that work: Prediction-based models (predict the expected value, score the error), LSTM autoencoders (learn context-dependent reconstruction), dynamic thresholds conditioned on time features.
Collective Anomalies
A sequence of observations is collectively anomalous, even though individual observations may be normal.
Example: In network intrusion detection, a port scan involves connecting to 1,024 different ports in sequence. Each individual connection looks like a normal network request. The pattern - the specific ordering and rapid succession - is the anomaly.
Example: In ECG analysis, a patient's heart rhythm shows a specific pattern of P-wave, QRS complex, and T-wave deformations across 30 consecutive beats. Each individual beat might not trigger a point alert, but the sustained pattern indicates a specific arrhythmia.
Why this is the hardest category: No single observation tells you anything. You need the model to build up evidence across multiple time steps and recognize the emergent pattern. This is exactly what recurrent and attention-based architectures are designed for.
Detectors that work: LSTM autoencoders (reconstruct full sequences), transformer-based models (TranAD, Anomaly Transformer), sliding-window collective scores.
Core Approaches - Four Families of Methods
Approach A: Statistical Methods
Statistical methods are your first line of defense. They are fast, interpretable, require no training data labels, and often catch 60–80% of the obvious anomalies before you ever need a neural network.
Rolling Z-Score
The rolling z-score asks: "Is this observation more than standard deviations away from the recent local mean?"
For a time series , with a rolling window of size :
Flag as anomaly if (typical: )
Why this beats a global z-score: A global z-score uses the mean and standard deviation of the entire series. If the series has a trend or seasonality, the global statistics are meaningless - observations from a high-trend period are always flagged; observations from low-trend periods are never flagged. The rolling window localizes the statistics to the recent past, adapting to the local regime.
Limitation: The window size is a critical hyperparameter. Too small: high variance in the statistics, noisy detector. Too large: slow to adapt to regime changes, misses contextual anomalies.
CUSUM (Cumulative Sum)
CUSUM is designed to detect sustained shifts in the mean. It accumulates evidence across time steps, firing when the cumulative deviation exceeds a limit.
The algorithm maintains two statistics:
detects upward shifts; detects downward shifts.
Where:
- is the target (expected) mean
- is the allowable slack (typically )
- Alert fires when or , where is the decision limit (typically to )
Why CUSUM catches what z-score misses: If the mean shifts from 0 to , no individual observation will cross a threshold. But CUSUM accumulates those small deviations. After enough steps, crosses and the alert fires. This is exactly the fraud scenario from the opening - small, consistent deviations from normal behavior.
When to Use Statistical Methods
Use statistical methods when:
- You need a fast, explainable baseline
- The anomalies are relatively large compared to noise
- You have limited compute budget
- You need to operate on streaming data with very low latency
Do not use statistical methods alone when:
- Anomalies are contextual (the z-score does not know about context)
- The series has complex multivariate dependencies
- Collective anomalies are the primary concern
Approach B: Reconstruction-Based Methods - LSTM Autoencoder
The reconstruction-based approach trains a model to compress and then reconstruct normal sequences. The intuition: a model trained only on normal data learns to reconstruct normal sequences well. When it sees an anomalous sequence, it does not know how to reconstruct it - the reconstruction error is high. High reconstruction error = anomaly.
Architecture
The LSTM autoencoder has two components:
Encoder: Takes a sequence of length with features. Processes it step by step with an LSTM. The final hidden state is the latent representation - a compressed encoding of the sequence.
Decoder: Takes the latent representation. Generates a reconstruction of the input sequence, step by step, running the LSTM in reverse or using a separate decoder LSTM.
Training objective: Minimize mean squared error between input sequence and reconstruction:
Inference: For a new window of length , compute the reconstruction error:
Flag as anomaly if .
Why This Works for Sequences
The LSTM encoder processes the entire sequence before producing the latent code. The decoder must reconstruct the full sequence from that code. If the sequence contains unusual transitions or patterns - the signature of a collective or contextual anomaly - the encoder cannot compress them efficiently into the learned latent space, and the decoder cannot reconstruct them from a code that was trained only on normal patterns.
This is fundamentally different from point anomaly detection: the anomaly score is computed over the entire sequence, not each individual point.
Approach C: Prediction-Based Methods
Prediction-based methods train a model to predict the next value (or the next few values) in a sequence. At inference time, a large prediction error signals an anomaly.
Why this is intuitive: A model that has learned normal sequence dynamics can predict the next step accurately when the sequence is behaving normally. When the sequence deviates from learned patterns - an anomalous transition occurs - the prediction error spikes.
Example implementation: Train an LSTM or Transformer to predict given . At inference time, compute . Apply a threshold to to generate anomaly flags.
Advantage over reconstruction-based: Prediction-based methods give point-in-time anomaly scores (the error at each step), making it easier to pinpoint when an anomaly occurred. Reconstruction-based methods give a score for the whole window.
Limitation: Prediction-based methods struggle with concept drift - if the underlying process changes (not an anomaly, but a regime change), the predictor's error spikes, causing false positives. You need drift detection and model retraining infrastructure.
Approach D: Density-Based Methods - Isolation Forest on Sequence Features
Isolation Forest can be applied to sequences by first extracting features from sliding windows and then applying the forest to those feature vectors.
Feature extraction from sequences (common features):
- Mean, standard deviation, min, max over the window
- Rate of change (first differences)
- Spectral features: dominant frequency from FFT, spectral entropy
- Autocorrelation at lag 1, lag 2
- Rolling skewness and kurtosis
These features are computed for each sliding window, giving a feature vector per window. Isolation Forest then scores each window's feature vector for anomalousness.
Why this is still useful in the deep learning era: Isolation Forest on sequence features is fast, requires no GPU, handles high-dimensional feature vectors well, and is interpretable (you can look at which features most contributed to isolation). It is a strong baseline and often competitive with LSTM approaches on datasets where anomalies are expressed in summary statistics rather than fine-grained sequence dynamics.
Isolation Forest recap (Liu et al., 2008): Build n isolation trees by randomly selecting a feature and a random split value. Anomalies are isolated in fewer splits because they are rare and different - their average path length across trees is shorter. The anomaly score is derived from the expected path length.
Threshold Selection - The Problem No One Talks About Enough
Building the anomaly detector is the easy part. Choosing the threshold that separates "anomaly" from "normal" is where production systems live or die.
Why Fixed Thresholds Fail
A fixed threshold of reconstruction_error > 0.05 works great during testing. Then:
- You deploy on Monday. False positive rate is 2%. Acceptable.
- Wednesday brings a software release - normal behavior shifts, reconstruction error distribution shifts up. False positive rate is now 15%.
- The on-call team starts ignoring alerts. Alert fatigue sets in.
- Friday: a real anomaly fires. The team, exhausted by false positives, acknowledges and closes it within 30 seconds without investigation. They miss the incident.
This is not hypothetical. It is the documented failure mode of fixed-threshold anomaly systems in production (see: Nair et al., "TimeCluster: Dimension Reduction Applied to Temporal Data for Visual Analytics," 2019, which catalogs alert fatigue patterns across multiple enterprise deployments).
Dynamic Thresholds
Dynamic thresholds adapt the decision boundary to the recent distribution of anomaly scores.
Rolling percentile threshold: Keep a rolling window of the last N anomaly scores (e.g., N = 10,000). Set the threshold at the 99th or 99.9th percentile of that rolling distribution. The threshold automatically adjusts as the score distribution shifts.
Seasonality-aware thresholds: If your series has daily or weekly seasonality, maintain separate threshold distributions for each time bucket (e.g., hour of day, day of week). A server at peak load on Monday at 9 AM gets compared to the Monday-9AM distribution, not the global distribution.
Exponentially weighted thresholds: Weight recent observations more heavily in computing the threshold. This allows the threshold to adapt quickly to regime changes while still being stable over short timescales.
Extreme Value Theory
Extreme Value Theory provides a principled statistical framework for setting thresholds on anomaly scores. The Peaks-Over-Threshold (POT) method, based on the Generalized Pareto Distribution, models the tail of the anomaly score distribution.
The intuition: You do not care about the bulk of the anomaly score distribution - you care about the tail. EVT tells you the probability of observing a score above any given value in the tail, even if you have never seen a score that high in your training data. The Generalized Pareto Distribution gives:
where is the pre-threshold, is the shape parameter, and is the scale parameter.
Implementation: Given a collection of anomaly scores, fit a Generalized Pareto Distribution to the scores above a pre-threshold . Use the fitted distribution to compute the threshold at a desired false positive rate (e.g., ):
Siffer et al. ("Anomaly Detection in Streams with Extreme Value Theory," 2017) showed that EVT-based thresholds outperform fixed and rolling-percentile thresholds on multiple real-world streaming anomaly detection benchmarks, because they correctly model the statistical behavior of extreme events rather than just empirically fitting to observed data.
NumPy From Scratch - Rolling Z-Score and CUSUM Anomaly Detectors
This implementation is self-contained and runnable. It covers both the rolling z-score detector and CUSUM, along with a synthetic test harness.
import numpy as np
from dataclasses import dataclass
import warnings
warnings.filterwarnings("ignore")
@dataclass
class RollingZScoreResult:
scores: np.ndarray # z-scores at each timestep
flags: np.ndarray # boolean array: True = anomaly
rolling_mean: np.ndarray
rolling_std: np.ndarray
def rolling_z_score_detector(
series: np.ndarray,
window: int = 50,
threshold: float = 3.0,
min_periods: int = 10,
) -> RollingZScoreResult:
"""
Detect anomalies using a rolling z-score.
Parameters
----------
series : 1-D array of observations
window : number of past observations for local statistics
threshold : z-score beyond which a point is flagged
min_periods : minimum observations before scoring begins
Returns
-------
RollingZScoreResult with scores, flags, and local statistics
"""
n = len(series)
z_scores = np.full(n, np.nan)
rolling_mean = np.full(n, np.nan)
rolling_std = np.full(n, np.nan)
for t in range(n):
# Use observations from max(0, t-window) to t (exclusive of t itself)
start = max(0, t - window)
window_data = series[start:t]
if len(window_data) < min_periods:
continue
mu = np.mean(window_data)
sigma = np.std(window_data, ddof=1)
rolling_mean[t] = mu
rolling_std[t] = sigma
if sigma < 1e-10:
# Constant series - any deviation is an anomaly
z_scores[t] = 0.0 if abs(series[t] - mu) < 1e-10 else np.inf
else:
z_scores[t] = (series[t] - mu) / sigma
flags = np.abs(z_scores) > threshold
# Do not flag NaN positions (insufficient history)
flags[np.isnan(z_scores)] = False
return RollingZScoreResult(
scores=z_scores,
flags=flags,
rolling_mean=rolling_mean,
rolling_std=rolling_std,
)
def cusum_detector(
series: np.ndarray,
target_mean: float,
slack_multiplier: float = 0.5,
decision_limit_multiplier: float = 4.0,
) -> dict:
"""
CUSUM detector for sustained mean shifts.
Parameters
----------
series : 1-D array of observations
target_mean : expected (in-control) mean
slack_multiplier : k = slack_multiplier * sigma_hat
decision_limit_multiplier : h = decision_limit_multiplier * sigma_hat
Returns
-------
dict with 's_pos', 's_neg', 'flags', 'sigma_hat'
"""
n = len(series)
# Estimate sigma from the full series (assumes mostly normal data)
sigma_hat = np.std(series, ddof=1)
k = slack_multiplier * sigma_hat # allowable slack
h = decision_limit_multiplier * sigma_hat # decision limit
s_pos = np.zeros(n)
s_neg = np.zeros(n)
flags = np.zeros(n, dtype=bool)
for t in range(1, n):
deviation = series[t] - target_mean
s_pos[t] = max(0.0, s_pos[t - 1] + deviation - k)
s_neg[t] = max(0.0, s_neg[t - 1] - deviation - k)
flags[t] = (s_pos[t] > h) or (s_neg[t] > h)
return {
"s_pos": s_pos,
"s_neg": s_neg,
"flags": flags,
"sigma_hat": sigma_hat,
"decision_limit": h,
"slack": k,
}
def generate_synthetic_series(
n: int = 500,
anomaly_fraction: float = 0.05,
seed: int = 42,
) -> tuple:
"""
Generate a synthetic time series with injected point and collective anomalies.
Returns (series, true_anomaly_labels)
"""
rng = np.random.default_rng(seed)
# Base signal: AR(1) process with slow seasonality
series = np.zeros(n)
series[0] = 0.0
phi = 0.7 # AR coefficient
for t in range(1, n):
series[t] = phi * series[t - 1] + rng.normal(0, 1.0)
# Add weekly seasonality
t_idx = np.arange(n)
series += 3.0 * np.sin(2 * np.pi * t_idx / 50)
true_labels = np.zeros(n, dtype=int)
# Inject point anomalies
n_anomalies = int(n * anomaly_fraction)
anomaly_positions = rng.choice(n, size=n_anomalies, replace=False)
for pos in anomaly_positions:
direction = rng.choice([-1, 1])
magnitude = rng.uniform(6.0, 10.0)
series[pos] += direction * magnitude
true_labels[pos] = 1
# Inject a collective anomaly: sustained shift for 10 steps
collective_start = int(n * 0.6)
series[collective_start: collective_start + 10] += 4.0
true_labels[collective_start: collective_start + 10] = 1
return series, true_labels
def evaluate_detector(true_labels: np.ndarray, predicted_flags: np.ndarray) -> dict:
"""Compute precision, recall, F1 for anomaly detector."""
tp = int(np.sum((predicted_flags == 1) & (true_labels == 1)))
fp = int(np.sum((predicted_flags == 1) & (true_labels == 0)))
fn = int(np.sum((predicted_flags == 0) & (true_labels == 1)))
precision = tp / (tp + fp + 1e-10)
recall = tp / (tp + fn + 1e-10)
f1 = 2 * precision * recall / (precision + recall + 1e-10)
return {
"precision": precision,
"recall": recall,
"f1": f1,
"tp": tp,
"fp": fp,
"fn": fn,
}
# ----- Main demonstration -----
if __name__ == "__main__":
series, true_labels = generate_synthetic_series(n=500, seed=42)
# Rolling z-score
result = rolling_z_score_detector(series, window=50, threshold=3.0)
z_metrics = evaluate_detector(true_labels, result.flags.astype(int))
print("Rolling Z-Score Results:")
print(f" Precision: {z_metrics['precision']:.3f}")
print(f" Recall: {z_metrics['recall']:.3f}")
print(f" F1: {z_metrics['f1']:.3f}")
print(f" TP: {z_metrics['tp']}, FP: {z_metrics['fp']}, FN: {z_metrics['fn']}")
print()
# CUSUM
target_mean = np.mean(series[:100]) # Use first 100 points as baseline
cusum_result = cusum_detector(series, target_mean=target_mean)
c_metrics = evaluate_detector(true_labels, cusum_result["flags"].astype(int))
print("CUSUM Results:")
print(f" Precision: {c_metrics['precision']:.3f}")
print(f" Recall: {c_metrics['recall']:.3f}")
print(f" F1: {c_metrics['f1']:.3f}")
print(f" TP: {c_metrics['tp']}, FP: {c_metrics['fp']}, FN: {c_metrics['fn']}")
print(f" Decision limit: {cusum_result['decision_limit']:.3f}")
Expected output (approximate):
Rolling Z-Score Results:
Precision: 0.741
Recall: 0.680
F1: 0.709
TP: 17, FP: 6, FN: 8
CUSUM Results:
Precision: 0.583
Recall: 0.560
F1: 0.571
TP: 14, FP: 10, FN: 11
Decision limit: 3.847
The rolling z-score does better here because the injected anomalies are large point spikes - exactly what z-score is designed for. CUSUM performs better on sustained mean shifts; try increasing the collective anomaly duration to 30 steps to see CUSUM pull ahead.
PyTorch Implementation - LSTM Autoencoder
This is a complete, runnable LSTM autoencoder for multivariate sequence anomaly detection.
import numpy as np
import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
from dataclasses import dataclass
# ──────────────────────────────────────────
# Dataset
# ──────────────────────────────────────────
class SequenceDataset(Dataset):
"""Sliding-window dataset of fixed-length sequences."""
def __init__(self, data: np.ndarray, seq_len: int):
"""
Parameters
----------
data : array of shape (T, d) - multivariate time series
seq_len : length of each sliding window
"""
self.data = torch.tensor(data, dtype=torch.float32)
self.seq_len = seq_len
def __len__(self):
return len(self.data) - self.seq_len + 1
def __getitem__(self, idx):
return self.data[idx : idx + self.seq_len]
# ──────────────────────────────────────────
# Model
# ──────────────────────────────────────────
class LSTMEncoder(nn.Module):
def __init__(self, input_dim: int, hidden_dim: int, num_layers: int, latent_dim: int):
super().__init__()
self.lstm = nn.LSTM(
input_size=input_dim,
hidden_size=hidden_dim,
num_layers=num_layers,
batch_first=True,
)
self.fc = nn.Linear(hidden_dim, latent_dim)
def forward(self, x: torch.Tensor) -> torch.Tensor:
# x: (batch, seq_len, input_dim)
_, (h_n, _) = self.lstm(x)
# h_n: (num_layers, batch, hidden_dim) - take last layer
last_hidden = h_n[-1] # (batch, hidden_dim)
latent = self.fc(last_hidden) # (batch, latent_dim)
return latent
class LSTMDecoder(nn.Module):
def __init__(
self,
latent_dim: int,
hidden_dim: int,
num_layers: int,
output_dim: int,
seq_len: int,
):
super().__init__()
self.seq_len = seq_len
self.num_layers = num_layers
self.fc = nn.Linear(latent_dim, hidden_dim)
self.lstm = nn.LSTM(
input_size=hidden_dim,
hidden_size=hidden_dim,
num_layers=num_layers,
batch_first=True,
)
self.output_projection = nn.Linear(hidden_dim, output_dim)
def forward(self, latent: torch.Tensor) -> torch.Tensor:
# latent: (batch, latent_dim)
h0 = self.fc(latent) # (batch, hidden_dim)
# Repeat latent representation across the sequence length as decoder input
decoder_input = h0.unsqueeze(1).repeat(1, self.seq_len, 1) # (batch, seq_len, hidden_dim)
# Initialize hidden state from latent
h_init = h0.unsqueeze(0).repeat(self.num_layers, 1, 1) # (num_layers, batch, hidden_dim)
c_init = torch.zeros_like(h_init)
output, _ = self.lstm(decoder_input, (h_init, c_init))
# output: (batch, seq_len, hidden_dim)
reconstruction = self.output_projection(output) # (batch, seq_len, output_dim)
return reconstruction
class LSTMAutoencoder(nn.Module):
def __init__(
self,
input_dim: int,
hidden_dim: int = 64,
latent_dim: int = 32,
num_layers: int = 2,
seq_len: int = 50,
):
super().__init__()
self.encoder = LSTMEncoder(input_dim, hidden_dim, num_layers, latent_dim)
self.decoder = LSTMDecoder(latent_dim, hidden_dim, num_layers, input_dim, seq_len)
def forward(self, x: torch.Tensor) -> torch.Tensor:
latent = self.encoder(x)
reconstruction = self.decoder(latent)
return reconstruction
# ──────────────────────────────────────────
# Trainer
# ──────────────────────────────────────────
@dataclass
class TrainingConfig:
seq_len: int = 50
hidden_dim: int = 64
latent_dim: int = 32
num_layers: int = 2
batch_size: int = 64
learning_rate: float = 1e-3
num_epochs: int = 30
device: str = "cpu"
def train_lstm_autoencoder(
train_data: np.ndarray,
config: TrainingConfig,
verbose: bool = True,
) -> LSTMAutoencoder:
"""
Train an LSTM autoencoder on normal sequences only.
Parameters
----------
train_data : array of shape (T, d) - contains normal sequences only
config : training hyperparameters
verbose : print loss per epoch
Returns
-------
Trained LSTMAutoencoder
"""
device = torch.device(config.device)
dataset = SequenceDataset(train_data, seq_len=config.seq_len)
loader = DataLoader(dataset, batch_size=config.batch_size, shuffle=True)
input_dim = train_data.shape[1]
model = LSTMAutoencoder(
input_dim=input_dim,
hidden_dim=config.hidden_dim,
latent_dim=config.latent_dim,
num_layers=config.num_layers,
seq_len=config.seq_len,
).to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=config.learning_rate)
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(
optimizer, T_max=config.num_epochs
)
criterion = nn.MSELoss()
model.train()
for epoch in range(config.num_epochs):
epoch_loss = 0.0
n_batches = 0
for batch in loader:
batch = batch.to(device)
optimizer.zero_grad()
reconstruction = model(batch)
loss = criterion(reconstruction, batch)
loss.backward()
# Gradient clipping prevents LSTM training instability
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
optimizer.step()
epoch_loss += loss.item()
n_batches += 1
scheduler.step()
if verbose and (epoch + 1) % 5 == 0:
avg_loss = epoch_loss / n_batches
print(f"Epoch {epoch + 1:3d}/{config.num_epochs} Loss: {avg_loss:.6f}")
return model
# ──────────────────────────────────────────
# Scoring
# ──────────────────────────────────────────
def compute_anomaly_scores(
model: LSTMAutoencoder,
test_data: np.ndarray,
seq_len: int,
batch_size: int = 256,
device: str = "cpu",
) -> np.ndarray:
"""
Compute per-timestep anomaly scores using reconstruction error.
Each timestep receives the average reconstruction error across
all windows it participates in. This gives smoother scores than
using only the window centered on the timestep.
Parameters
----------
model : trained LSTMAutoencoder
test_data : array of shape (T, d)
seq_len : window length (must match training)
batch_size : inference batch size
device : torch device string
Returns
-------
anomaly_scores : array of shape (T,) - higher = more anomalous
"""
_device = torch.device(device)
model.eval()
model.to(_device)
dataset = SequenceDataset(test_data, seq_len=seq_len)
loader = DataLoader(dataset, batch_size=batch_size, shuffle=False)
T = len(test_data)
score_accumulator = np.zeros(T)
count_accumulator = np.zeros(T)
window_idx = 0
with torch.no_grad():
for batch in loader:
batch = batch.to(_device)
reconstruction = model(batch)
# Per-element MSE: (batch, seq_len, d) -> (batch, seq_len)
per_step_error = ((batch - reconstruction) ** 2).mean(dim=-1).cpu().numpy()
for i in range(len(per_step_error)):
start = window_idx + i
end = start + seq_len
score_accumulator[start:end] += per_step_error[i]
count_accumulator[start:end] += 1.0
window_idx += len(per_step_error)
# Avoid division by zero at boundaries
count_accumulator = np.maximum(count_accumulator, 1.0)
anomaly_scores = score_accumulator / count_accumulator
return anomaly_scores
def select_threshold_percentile(
scores: np.ndarray,
percentile: float = 99.0,
) -> float:
"""Set threshold at the given percentile of scores on a validation set."""
return float(np.percentile(scores, percentile))
# ──────────────────────────────────────────
# End-to-end demonstration
# ──────────────────────────────────────────
def generate_multivariate_series(n: int = 1000, d: int = 3, seed: int = 0) -> tuple:
"""Returns (train_data, test_data, test_labels). Train is purely normal."""
rng = np.random.default_rng(seed)
# Correlated multivariate AR(1) process
A = np.array([[0.7, 0.1, 0.0],
[0.0, 0.6, 0.2],
[0.0, 0.0, 0.8]])
noise_std = 0.5
def generate_ar(n_steps):
data = np.zeros((n_steps, d))
for t in range(1, n_steps):
data[t] = A @ data[t - 1] + rng.normal(0, noise_std, d)
return data
train_data = generate_ar(n)
test_data = generate_ar(n)
test_labels = np.zeros(n, dtype=int)
# Inject collective anomaly: positions 200-203
for pos in [200, 201, 202, 203]:
test_data[pos] += rng.uniform(4.0, 6.0, d)
test_labels[pos] = 1
# Inject point anomaly at 400
test_data[400, 0] += 8.0
test_labels[400] = 1
# Inject contextual anomaly at 600-601 (rapid reversal)
test_data[600] -= 5.0
test_data[601] += 5.0
test_labels[600] = 1
test_labels[601] = 1
return train_data, test_data, test_labels
if __name__ == "__main__":
torch.manual_seed(42)
np.random.seed(42)
print("Generating synthetic multivariate data...")
train_data, test_data, test_labels = generate_multivariate_series(n=1000, d=3)
config = TrainingConfig(
seq_len=30,
hidden_dim=32,
latent_dim=16,
num_layers=1,
batch_size=64,
learning_rate=1e-3,
num_epochs=20,
device="cpu",
)
print("Training LSTM autoencoder on normal data only...")
model = train_lstm_autoencoder(train_data, config, verbose=True)
print("\nComputing anomaly scores on test data...")
scores = compute_anomaly_scores(model, test_data, seq_len=config.seq_len)
# Calibrate threshold on training scores (all normal)
train_scores = compute_anomaly_scores(model, train_data, seq_len=config.seq_len)
threshold = select_threshold_percentile(train_scores, percentile=99.0)
print(f"Threshold (99th percentile of train scores): {threshold:.6f}")
predicted = (scores > threshold).astype(int)
tp = int(np.sum((predicted == 1) & (test_labels == 1)))
fp = int(np.sum((predicted == 1) & (test_labels == 0)))
fn = int(np.sum((predicted == 0) & (test_labels == 1)))
precision = tp / (tp + fp + 1e-10)
recall = tp / (tp + fn + 1e-10)
f1 = 2 * precision * recall / (precision + recall + 1e-10)
print(f"\nResults:")
print(f" TP: {tp}, FP: {fp}, FN: {fn}")
print(f" Precision: {precision:.3f}")
print(f" Recall: {recall:.3f}")
print(f" F1: {f1:.3f}")
Key design decisions in this implementation:
-
Per-timestep score via window averaging. Rather than giving each 30-step window a single score, we accumulate reconstruction error at each timestep across all windows it participates in. This produces a smooth, continuous anomaly score time series - much easier to threshold than window-level scores.
-
Train only on normal data. The autoencoder never sees anomalous sequences during training. This is essential - if you include anomalies in training, the model learns to reconstruct them too, and your reconstruction error scores become meaningless.
-
Gradient clipping. LSTM training is sensitive to exploding gradients. The
clip_grad_norm_(model.parameters(), max_norm=1.0)call prevents training instability. -
Threshold from train scores. Setting the threshold at the 99th percentile of reconstruction errors on training data (which is purely normal) gives a principled starting point. This corresponds to a 1% false positive rate on the training distribution.
Anomaly Detection Pipeline - System Architecture
The pipeline has two feedback loops:
Loop 1 (alert quality): Alert deduplication suppresses repeated alerts for the same anomaly. Without this, a single anomalous sequence of 100 timesteps generates 100 alerts - all referring to the same event.
Loop 2 (model freshness): The score distribution is continuously monitored. When the distribution drifts significantly (the normal reconstruction error baseline shifts), the drift detector triggers a retraining job. The retrained model is deployed via canary - serving 5% of traffic first, with metrics comparison against the champion model before full rollout.
Production Engineering Notes
Handling Concept Drift
Concept drift is the gradual or sudden change in the underlying data distribution. In anomaly detection, it is especially insidious: if the normal process changes, your model's reconstruction error baseline shifts upward for normal data, causing a flood of false positives.
Detection: Monitor the distribution of anomaly scores on data that has been confirmed normal (e.g., by human review or downstream signal - no incidents were filed). Use a two-sample test (Kolmogorov-Smirnov, Maximum Mean Discrepancy) between the current score distribution and the training-time baseline. Set a drift alarm when the test statistic exceeds a threshold.
Response strategies:
- Threshold adaptation: Adjust the anomaly threshold upward to match the new normal score distribution. Fast, does not require retraining, but only works for gradual drift.
- Online fine-tuning: Continuously fine-tune the model on a rolling buffer of confirmed-normal recent data. Works for slow drift.
- Full retraining: For sudden regime changes (major software deployment, hardware replacement, seasonal shift), retrain from scratch on recent data.
Class Imbalance - Anomalies Are Rare
In most real systems, the anomaly rate is 0.01% to 1%. This creates a fundamental challenge: a detector that flags everything as normal achieves 99%+ accuracy on raw accuracy metrics while being completely useless.
Do not optimize for accuracy. Use precision, recall, and F1 on the anomaly class. Or use AUROC (Area Under the ROC Curve) and AUPRC (Area Under the Precision-Recall Curve). AUPRC is more informative than AUROC when classes are highly imbalanced.
During training: For reconstruction-based methods, imbalance is handled naturally - you train only on normal data. For supervised methods, use class weighting or oversampling (SMOTE on sequence feature vectors, or synthetic anomaly injection - see Schlegel et al., "A Comparative Study of Deep Learning Approaches for Anomaly Detection in IoT Networks," 2021).
Synthetic anomaly injection: During training, randomly corrupt windows with point anomalies, collective shifts, or label flips. This teaches the model to explicitly reconstruct normal sequences and reject corrupted ones. TranAD uses a related technique (adversarial perturbation during training) to sharpen the reconstruction error boundary.
Alert Fatigue - The Operational Killer
A detector with 5% false positive rate sounds fine statistically. In practice, on a system that processes 1 million events per hour, that is 50,000 false alarms per hour. No on-call team survives this.
Practical targets:
- Fewer than 10 alerts per on-call shift for high-severity issues
- For lower-severity alerts: daily digest rather than real-time paging
- Alert deduplication: consecutive anomalous windows from the same event count as one alert (merge windows within 5 minutes, for example)
Tiered alert severity: Not all anomalies are equal. Map anomaly score magnitude to severity tiers. Only page on-call for the highest tier. Lower tiers go to a dashboard or daily report.
Human-in-the-loop feedback: Build an interface for on-call engineers to mark alerts as true positive or false positive. Use this feedback to retrain or fine-tune the detector. Systems that incorporate human feedback improve rapidly - false positive rates typically drop 40–60% after 2–3 weeks of feedback collection.
Evaluation Metrics for Anomaly Detection
Precision at k (P@k): Of the k highest-scored events, what fraction are true anomalies? Use this when you have a fixed investigation capacity - if your team can review 50 alerts per day, P@50 is the metric that matters.
F1 at threshold: Standard F1 computed at a specific threshold. Useful for comparing detectors when you have fixed the operating point. Always report the threshold along with the F1 - an F1 of 0.9 at a threshold that generates 50,000 false positives per day is not useful.
Time-tolerant evaluation: For sequence anomalies, a detection within plus or minus 5 timesteps of the true anomaly onset is often acceptable. Standard precision/recall penalizes a detector that fires 3 timesteps late equally to one that misses the anomaly entirely. Use point-adjust evaluation (Xu et al., 2018) which gives credit for detections that overlap with any anomaly window.
AUPRC (Area Under the Precision-Recall Curve): Integrates over all possible thresholds. Unlike AUROC, AUPRC degrades appropriately when anomalies are rare. Use this for final model comparison.
Common Mistakes
:::danger Critical Mistake: Including Anomalies in Autoencoder Training Data If your training set contains anomalous sequences, the autoencoder learns to reconstruct them. Reconstruction error for anomalies will be low - indistinguishable from normal data. Your detector becomes useless.
Always curate your training set to contain only confirmed-normal sequences. If you cannot guarantee this (unsupervised setting with no labels), accept some contamination and use a contamination-robust training objective, or apply an initial coarse filter (z-score) to remove obvious outliers before training the autoencoder. :::
:::danger Critical Mistake: Evaluating with a Fixed Global Threshold Chosen on Test Data Setting your threshold by looking at what value gives the best F1 on the test set is a severe form of data leakage. The threshold must be chosen on a held-out validation set (or on training-time score distributions) without access to test labels. Otherwise your reported metrics are optimistic to the point of being meaningless. :::
:::warning Mistake: Ignoring the Temporal Gap Between Train and Test Sets
If your training data ends on January 1 and your test data starts on January 2, you are not accounting for the model's warm-up period - the initial sequence of test data where the model has no context from the previous series. For LSTM models, the initial hidden state is zero; the first few hundred steps of test data will have elevated reconstruction error simply because the model is "cold starting." Either discard the first seq_len * 3 timesteps of test scores, or warm up the model by running it forward on the last seq_len * 3 steps of training data before evaluating on test.
:::
:::warning Mistake: Using MSE as Your Only Reconstruction Loss MSE heavily penalizes large errors, which sounds desirable for anomaly detection. But it also means the model concentrates its capacity on reconstructing the high-variance dimensions of your multivariate series, essentially ignoring low-variance dimensions. If anomalies manifest as subtle deviations in a normally low-variance channel, MSE-trained autoencoders will miss them. Consider normalized MSE (divide by per-channel variance), or use MAE combined with MSE. :::
:::warning Mistake: Not Accounting for Seasonality in Threshold Setting Setting a single threshold from the 99th percentile of all reconstruction scores ignores seasonality. If reconstruction error is systematically higher on weekday mornings (because of bursty traffic patterns), your threshold will be exceeded every Monday at 9 AM - by perfectly normal data. Use time-aware thresholds: compute the 99th percentile separately for each hour-of-week bucket. :::
:::danger Critical Mistake: Treating Anomaly Detection as Supervised Classification With Almost No Labels Many practitioners with limited label data try to frame anomaly detection as supervised binary classification (positives = anomalies, negatives = normal). With a 0.1% anomaly rate and 1,000 training samples, you have 1 anomaly. No classifier can learn from one positive example.
Use unsupervised methods (autoencoder, Isolation Forest, CUSUM) and invest your labeling budget in curating a high-quality evaluation set instead of a training set. The evaluation set is where labels matter most - it lets you tune your threshold and measure true performance. :::
Interview Questions and Answers
Q1: What is the difference between a point anomaly, a contextual anomaly, and a collective anomaly? Give a concrete example of each.
Answer:
A point anomaly is a single observation that is globally unusual regardless of context. Example: a server CPU reading of 100% when the machine is in a cold standby pool with no workload assigned. The value is extreme enough that it is anomalous without needing to compare it to its neighbors.
A contextual anomaly is an observation that is anomalous given its context but would be normal in a different context. Example: a CPU reading of 85% is completely normal during a peak batch job at 2 PM on a weekday, but is anomalous at 3 AM Sunday when the machine should be idle. The value 85% is not globally extreme - the context makes it anomalous. Standard point detectors miss this because they do not model the context.
A collective anomaly is a sequence of observations that is collectively anomalous, even though each individual observation might appear normal. Example: in network security, a port scan consists of 1,024 sequential connection attempts to different ports. Each individual connection looks like a normal network request. The anomaly only becomes visible when you analyze the sequence as a whole and recognize the scanning pattern.
Collective anomalies are the hardest to detect and require models that explicitly reason about temporal structure - LSTM autoencoders, transformer-based detectors, or rule-based sliding-window aggregators.
Q2: An LSTM autoencoder for anomaly detection has a very high false positive rate in production. What are the three most likely causes and how do you diagnose each?
Answer:
Cause 1: The threshold is too low. The threshold was set on a validation set but the production score distribution is different (either due to distributional shift or because the validation set was not representative). Diagnose by plotting the distribution of reconstruction scores on confirmed-normal production data and comparing it to the training-time distribution. If the production distribution is shifted right, the threshold needs to be adjusted upward.
Cause 2: The training set contained anomalies or non-representative normal data. If anomalous sequences leaked into training, the autoencoder learned to reconstruct both normal and anomalous patterns, blurring the reconstruction error boundary. Diagnose by inspecting the training set - look at the distribution of your anomaly scores on the training data itself. If the 99th percentile of training reconstruction errors is nearly as high as your test set anomaly scores, your training data is contaminated.
Cause 3: Concept drift - the normal distribution has shifted since training. If the system you are monitoring changed (software deployment, hardware replacement, user behavior shift), the new "normal" sequences differ from what the autoencoder was trained on. It reconstructs the new normal poorly, giving high error on legitimate data. Diagnose by monitoring the median and 90th percentile of reconstruction scores over time on data confirmed normal. A sustained upward trend signals drift.
Q3: Why does CUSUM catch anomalies that rolling z-score misses? When is rolling z-score the better choice?
Answer:
CUSUM catches sustained small mean shifts that no individual observation would exceed a z-score threshold for. If the mean shifts from 0 to 0.5 sigma - a modest change - individual observations still mostly fall within 3 sigma of the local mean. But CUSUM accumulates the deviation over time: after 20 steps of consistent positive deviation, the cumulative sum has grown large enough to cross the decision limit.
The rolling z-score catches sharp point anomalies - sudden large spikes that immediately exceed the threshold. It does not accumulate evidence, so small persistent deviations never trigger it.
Rolling z-score is the better choice when:
- Anomalies are large, sudden, and short-duration (a sensor glitch, a data corruption event)
- You need per-point anomaly flags rather than event-level detection
- Interpretability matters - explaining "this point is 4.2 standard deviations from the recent mean" is simpler than explaining CUSUM state
CUSUM is better when:
- Anomalies manifest as gradual drift or sustained low-magnitude shifts
- You are looking for process control failures (the manufacturing use case it was designed for)
- Early warning of slow degradation is the priority
In practice, run both in parallel and combine their alerts.
Q4: How do you evaluate an anomaly detection system when you have very few labeled anomalies?
Answer:
Use AUPRC, not accuracy. With a 0.1% anomaly rate, accuracy is meaningless. AUPRC measures the area under the precision-recall curve across all thresholds, giving a threshold-independent measure of ranking quality. Random performance corresponds to AUPRC equal to the anomaly rate (e.g., 0.001 for 0.1% anomaly rate), making it easy to see if your system is doing better than chance.
Use point-adjust evaluation for sequence anomalies. If your detector fires 3 timesteps after an anomaly begins, it should still get credit. Standard precision/recall counts a late detection as a false negative. Point-adjust evaluation (Xu et al., "DONUT: Unsupervised Anomaly Detection via Variational Autoencoder for Seasonal KPIs," 2018) credits any detection within the anomaly window.
Augment your label set with synthetic anomalies. You have few real labels, but you can inject synthetic anomalies into normal data (large spikes, level shifts, sudden pattern changes) and evaluate on those. This gives you a much larger evaluation set. Caveat: your synthetic anomalies may not match the distribution of real anomalies - use this to measure relative performance across methods, not absolute performance.
Use downstream business metrics when possible. For a fraud detection system, the best metric might be "dollars of fraud caught" rather than F1 on a sample. If you can instrument the full pipeline - detector outputs to investigator review to confirmed fraud cases - you get ground truth aligned with what actually matters.
Use temporal cross-validation, not k-fold. Standard cross-validation assumes i.i.d. data. For time series, you must use temporal cross-validation: train on [0, t], validate on [t, t+k], move forward. With few anomalies, some folds may have zero anomalies in the validation window - account for this in your evaluation.
Q5: A team proposes replacing your LSTM autoencoder with a much simpler Isolation Forest applied to sliding-window features. What are the tradeoffs and when would you agree to the replacement?
Answer:
Arguments for Isolation Forest on window features:
Speed and compute: Isolation Forest inference is orders of magnitude faster than LSTM forward passes. On a 100-feature window, scoring 1 million windows per second on CPU is feasible with Isolation Forest; doing the same with an LSTM autoencoder requires significant GPU resources.
Interpretability: Feature importance in Isolation Forest is directly available - you can tell an engineer "this window was flagged because the rolling standard deviation and spectral entropy were both in the top 1% of historical values." LSTM reconstruction error is harder to explain.
Robustness to distribution shift: Isolation Forest is based on random partitioning, which degrades more gracefully under moderate distribution shift than a learned deep model.
Arguments for keeping the LSTM autoencoder:
Collective anomalies in fine-grained temporal structure are invisible to feature-level methods. If your anomalies are patterns in the sequence dynamics - the ordering of events, specific transition probabilities - that cannot be captured by summary statistics (mean, std, autocorrelation at lag 1–3), Isolation Forest will miss them entirely.
When I would agree to the replacement:
If the team shows empirically, on a held-out test set with labeled anomalies, that Isolation Forest achieves comparable AUPRC and F1 to the LSTM autoencoder, and if the latency requirements favor the simpler model (real-time scoring at high throughput), then the Isolation Forest is the better engineering choice. Simpler systems are easier to maintain, debug, and operate. The best model is the simplest one that meets the requirements - not the most architecturally sophisticated.
I would insist on the empirical comparison covering at least two months of production data, not just a static test set, to account for the Isolation Forest's potential brittleness to distribution shift.
Q6: You are building an anomaly detection system for a payment processor. How would you handle the cold start problem for new user accounts?
Answer:
The cold start problem: new accounts have no transaction history. Your LSTM autoencoder, trained on sequences of length 50, cannot produce meaningful scores for accounts with fewer than 50 transactions. Your rolling z-score has no historical baseline to compute statistics against.
Strategy 1: Cohort-based models. Rather than building per-user models, train models on cohorts of similar users (by demographics, sign-up channel, initial transaction patterns). New users are assigned to a cohort and evaluated against cohort norms. This gives you reasonable anomaly scores from the first transaction.
Strategy 2: Hybrid scoring. For the first N transactions, use a purely feature-based point classifier that does not require sequence context (transaction amount vs. cohort distribution, merchant category match, device fingerprint). As history accumulates (once a user crosses 30 transactions, for example), gradually blend in the sequence-aware score:
where decreases as transaction count increases.
Strategy 3: Progressive context. Modify the LSTM autoencoder to handle variable-length sequences using padding and masking. For a new user with only 5 transactions, the model processes a padded sequence of length 50 with a mask that tells the LSTM which timesteps are real data. The reconstruction error is computed only on real timesteps. This works reasonably well once you have 10+ transactions, but the scores are noisier than full-history sequences.
Strategy 4: Transfer learning with adaptation. Fine-tune a global model (trained on all users) on the new account's growing transaction history using online learning. This allows the model to specialize to the individual while still having a reasonable prior from day one.
In practice, combine strategies 1 and 2: cohort priors for day one, progressive blending toward sequence-aware scoring as history accumulates.
:::tip 🎮 Interactive Playground
Visualize this concept: Try the Anomaly Detection Methods demo on the EngineersOfAI Playground - no code required.
:::