Adaptive Conditional Forest Sampling for Spectral Risk Optimisation under Decision-Dependent Uncertainty
| Authors | Marcell T. Kurbucz |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2603.12507 |
| Download | |
| Categories | cs.LG, stat.CO, stat.ML |
Abstract
Minimising a spectral risk objective, defined as a convex combination of expected cost and Conditional Value-at-Risk (CVaR), is challenging when the uncertainty distribution is decision-dependent, making both surrogate modelling and simulation-based ranking sensitive to tail estimation error. We propose Adaptive Conditional Forest Sampling (ACFS), a four-phase simulation-optimisation framework that integrates Generalised Random Forests for decision-conditional distribution approximation, CEM-guided global exploration, rank-weighted focused augmentation, and surrogate-to-oracle two-stage reranking before multi-start gradient-based refinement. We evaluate ACFS on two structurally distinct data-generating processes: a decision-dependent Student-t copula and a Gaussian copula with log-normal marginals, across three penalty-weight configurations and 100 replications per setting. ACFS achieves the lowest median oracle spectral risk on the second benchmark in every configuration, with median gaps over GP-BO ranging from 6.0% to 20.0%. On the first benchmark, ACFS and GP-BO are statistically indistinguishable in median objective, but ACFS reduces cross-replication dispersion by approximately 1.8 to 1.9 times on the first benchmark and 1.7 to 2.0 times on the second, indicating materially improved run-to-run reliability. ACFS also outperforms CEM-SO, SGD-CVaR, and KDE-SO in nearly all settings, while ablation and sensitivity analyses support the contribution and robustness of the proposed design.
Engineering Breakdown
Plain English
This paper addresses the problem of optimizing systems where the uncertainty distribution depends on the decisions you make (decision-dependent risk), which breaks standard surrogate modeling and simulation approaches. The authors propose ACFS, a four-phase framework that combines generalized random forests for approximating conditional distributions, cross-entropy method (CEM) for global search, intelligent sample augmentation weighted by ranking quality, and two-stage reranking before gradient refinement. The approach is evaluated on two structurally different synthetic problems: one using Student-t copulas and another using Gaussian copulas with log-normal marginals, demonstrating robustness to tail estimation errors that typically plague spectral risk minimization.
Core Technical Contribution
The core novelty is decomposing decision-dependent distribution approximation into a principled four-phase pipeline that explicitly handles tail risk estimation through rank-weighted sampling and surrogate-to-oracle validation. Traditional CVaR optimization assumes the distribution is fixed; this work handles cases where your decisions alter what future uncertainties look like, which is realistic in portfolio optimization, reliability engineering, and supply chain design. The key insight is that you can't trust surrogate models naively on tail quantities (the extreme losses), so ACFS uses ranking-weighted augmentation to focus computation where uncertainty has highest impact on risk, followed by a two-stage reranking that compares surrogate predictions against oracle evaluations before committing to a solution.
How It Works
ACFS operates in four sequential phases. First, Generalized Random Forests estimates the conditional distribution of outcomes given your current decision parameters, capturing how the distribution's shape changes as decisions change. Second, the Cross-Entropy Method performs global exploration of the decision space by iteratively sampling from a distribution over decisions, evaluating them, keeping the best performers, and shrinking the sampling distribution toward high-performing regions. Third, rank-weighted focused augmentation adds new simulation samples preferentially to decisions where the surrogate model's ranking of candidate solutions differs most from ground truth or where CVaR is most sensitive. Fourth, multi-start gradient-based refinement takes the best candidates from the surrogate and polishes them using oracle evaluations (expensive true simulations) to catch cases where the surrogate made errors. The output is a decision that minimizes the spectral risk objective (weighted combination of expected cost and CVaR) while accounting for decision-dependent tail behavior.
Production Impact
For engineers building risk-sensitive optimization systems (financial portfolios, safety-critical engineering, supply chain logistics), this approach directly addresses a critical gap: most tools assume fixed distributions and fail when your decisions change the distribution of outcomes you'll face. Adoption would mean integrating random forest-based distribution estimators upstream of your surrogate model, implementing CEM-guided search instead of pure Bayesian optimization or genetic algorithms, and adding a validation layer that compares surrogate rankings to actual simulations before deployment. The trade-off is computational cost—this requires training a distribution model, running CEM iterations, and then executing expensive oracle evaluations on candidates; the payoff is that you catch tail risks that naive surrogates miss, reducing the chance of choosing a decision that looks good on average but has catastrophic downside. Integration complexity is moderate: you need differentiable simulators for the refinement phase, and you must have a clear definition of decision-dependent uncertainty in your domain.
Limitations and When Not to Use This
The framework is evaluated only on synthetic, relatively low-dimensional problems (the abstract mentions Student-t and Gaussian copulas but doesn't specify dimensionality or compare to baselines quantitatively). A critical assumption is that Generalized Random Forests can accurately approximate decision-conditional distributions in your domain; if the true relationship between decisions and uncertainty is highly nonlinear, multimodal, or high-dimensional, this approximation may fail silently. The approach requires substantial oracle budget (expensive simulations) for the refinement phase, which may be prohibitive in domains where simulation cost is very high; the paper doesn't provide guidance on how much oracle budget you need or how to set it adaptively. Finally, there's no comparison to simpler baselines (standard CVaR optimization with fixed distributions, robust optimization, or Bayesian optimization with heteroscedastic surrogates), so it's unclear how much the added complexity actually improves over prior methods.
Research Context
This work builds on decades of research in stochastic optimization (CVaR and spectral risk measures), random forest methods for conditional distribution estimation, and simulation-based optimization under uncertainty. It extends beyond classical CVaR minimization (which assumes a known or pre-sampled distribution) into the decision-dependent setting, a less-studied but practically important regime. The paper sits at the intersection of distributionally-robust optimization and surrogate-assisted optimization, drawing from both communities to handle tail risk in a computationally tractable way. Open research questions include: how does performance degrade with dimensionality, how sensitive is the method to random forest hyperparameters, and can this approach scale to problems where oracle evaluations cost thousands of dollars each.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::