Competition-Aware CPC Forecasting with Near-Market Coverage
| Authors | Sebastian Frey et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2603.13059 |
| Download | |
| Categories | cs.LG, cs.AI |
Abstract
Cost-per-click (CPC) in paid search is a volatile auction outcome generated by a competitive landscape that is only partially observable from any single advertiser's history. Using Google Ads auction logs from a concentrated car-rental market (2021--2023), we forecast weekly CPC for 1,811 keyword series and approximate latent competition through complementary signals derived from keyword text, CPC trajectories, and geographic market structure. We construct (i) semantic neighborhoods and a semantic keyword graph from pretrained transformer-based representations of keyword text, (ii) behavioral neighborhoods via Dynamic Time Warping (DTW) alignment of CPC trajectories, and (iii) geographic-intent covariates capturing localized demand and marketplace heterogeneity. We extensively evaluate these signals both as stand-alone covariates and as relational priors in spatiotemporal graph forecasters, benchmarking them against strong statistical, neural, and time-series foundation-model baselines. Across methods, competition-aware augmentation improves stability and error profiles at business-relevant medium and longer horizons, where competitive regimes shift and volatility is most consequential. The results show that broad market-outcome coverage, combined with keyword-derived semantic and geographic priors, provides a scalable way to approximate latent competition and improve CPC forecasting in auction-driven markets.
Engineering Breakdown
Plain English
This paper tackles the problem of forecasting cost-per-click (CPC) in Google Ads auctions, where prices fluctuate based on hidden competitive dynamics that no single advertiser can fully observe. Using 2+ years of auction data from 1,811 keywords in the car-rental market, the authors built a forecasting system that reconstructs the latent competition landscape using three complementary signal sources: semantic similarity between keywords (via transformer embeddings), behavioral similarity based on CPC price movement patterns (via Dynamic Time Warping), and geographic demand variation. The core insight is that you can infer competitor behavior and market structure without directly observing competitors—by analyzing keyword semantics, price trajectories, and geographic intent together. This approach generates semantic neighborhoods, behavioral neighborhoods, and localized demand covariates that feed into a forecasting model, achieving measurable improvements in CPC prediction accuracy.
Core Technical Contribution
The key novelty is using multi-view neighborhood construction to approximate unobservable competition in auction markets. Rather than treating each keyword in isolation, the authors explicitly model three latent competition signals: (1) semantic neighbors (keywords that functionally compete) via pretrained transformer embeddings of keyword text, (2) behavioral neighbors (keywords with similar price dynamics) via Dynamic Time Warping distance on CPC trajectories, and (3) geographic-intent structure capturing localized market heterogeneity. This is different from prior CPC forecasting work because it acknowledges that competition is partially observable through keyword semantics and price behavior, and it combines these signals in a principled way. The geographic-intent covariates are particularly novel—they capture that demand and competition vary by region, allowing the model to distinguish between truly competitive keywords and those that just happen to move together.
How It Works
The system operates in three parallel streams that converge into a forecasting model. First, keyword text is encoded using a pretrained transformer (likely BERT-style) to create dense semantic representations; these embeddings are used to construct a semantic keyword graph where edges connect functionally similar keywords (e.g., 'car rentals NYC' and 'NYC vehicle rentals'). Second, the weekly CPC time series for each keyword is aligned against all other keywords using Dynamic Time Warping—a distance metric that captures shape similarity even if prices don't move at the same scale or timing—creating behavioral neighborhoods of keywords with correlated price movements. Third, geographic-intent covariates are engineered to capture demand hotspots and marketplace structure, likely including region-level search volume, competitor density, and seasonal patterns. All three signal streams (semantic neighbors, behavioral neighbors, geographic covariates) are then fed into a forecasting model—likely a regression or neural network architecture—that learns to weight these signals and predict next week's CPC for each keyword. The model leverages the observation that if a keyword is in your semantic neighborhood, its historical price behavior contains information about your future price, even though you've never competed directly.
Production Impact
For teams running paid search infrastructure, this approach directly improves bid optimization and budget forecasting accuracy. If you implement this system, you move from treating each keyword as an independent forecasting problem to jointly modeling 1,800+ keywords via shared semantic and behavioral structure—this reduces overfitting and allows transfer learning between keywords with similar competitive profiles. In production, this means lower forecasting error → better ROI predictions → more efficient bid allocation. The trade-offs are non-trivial: you need to maintain an embeddings model (compute cost for encoding all keywords regularly), compute DTW distances at scale (quadratic in keyword count without approximation), and engineer geographic-intent features for every market you operate in. Latency-wise, real-time inference is feasible if you precompute neighborhoods weekly, but rebuilding neighborhoods requires batch processing all keywords. For an advertiser with thousands of keywords across multiple markets, this adds engineering complexity but pays for itself through improved forecast accuracy and reduced wasteful bidding on high-CPC keywords you can now predict.
Limitations and When Not to Use This
The paper is evaluated on a single vertical (car-rental) in concentrated markets over a 2-year window, so generalization to other auction dynamics (e.g., retail, finance, B2B) is unproven—different verticals have different competitive structures and seasonality patterns. The approach assumes that semantic similarity and behavioral similarity are good proxies for latent competition, but this breaks down if competitors use radically different keyword strategies or if keyword semantics don't map cleanly to advertiser intent. DTW-based behavioral neighborhoods require sufficient historical data to be reliable; for new keywords or new advertiser entering the market, the system will have cold-start problems because the DTW distances are unreliable. The paper doesn't address how to handle sudden competitive shocks (a new entrant flooding the market, a competitor exiting) or algorithm changes by the auction mechanism itself—these would violate the stationarity assumptions baked into DTW alignment and time-series forecasting. Finally, the paper is incomplete in the abstract provided; we don't see the full evaluation results, which would be critical to understand real-world accuracy gains and whether the added complexity is justified empirically.
Research Context
This work sits at the intersection of auction theory, time-series forecasting, and representation learning. It builds on prior research in price forecasting for search advertising (e.g., Varian's work on auction equilibria) but adds modern ML tools—pretraining and embeddings—to surface latent competition. The approach is motivated by recent advances in using transformer embeddings for semantic similarity tasks, combined with classical time-series techniques (DTW) to find behavioral patterns. It contributes to the growing body of work on multi-view learning, where different data modalities (text, time series, geographic metadata) are combined to solve a single problem. The paper likely opens research directions on: (1) more sophisticated neighborhood aggregation methods beyond simple similarity thresholds, (2) handling non-stationary competition dynamics, (3) extending to other auction-based markets (ads, energy, real estate) where competition is latent, and (4) joint modeling of advertiser and publisher behavior in the same framework.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::