SurvHTE-Bench: A Benchmark for Heterogeneous Treatment Effect Estimation in Survival Analysis
| Authors | Shahriar Noroozizadeh et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2603.05483 |
| Download | |
| Categories | cs.LG, cs.AI, stat.ML |
Abstract
Estimating heterogeneous treatment effects (HTEs) from right-censored survival data is critical in high-stakes applications such as precision medicine and individualized policy-making. Yet, the survival analysis setting poses unique challenges for HTE estimation due to censoring, unobserved counterfactuals, and complex identification assumptions. Despite recent advances, from Causal Survival Forests to survival meta-learners and outcome imputation approaches, evaluation practices remain fragmented and inconsistent. We introduce SurvHTE-Bench, the first comprehensive benchmark for HTE estimation with censored outcomes. The benchmark spans (i) a modular suite of synthetic datasets with known ground truth, systematically varying causal assumptions and survival dynamics, (ii) semi-synthetic datasets that pair real-world covariates with simulated treatments and outcomes, and (iii) real-world datasets from a twin study (with known ground truth) and from an HIV clinical trial. Across synthetic, semi-synthetic, and real-world settings, we provide the first rigorous comparison of survival HTE methods under diverse conditions and realistic assumption violations. SurvHTE-Bench establishes a foundation for fair, reproducible, and extensible evaluation of causal survival methods. The data and code of our benchmark are available at: https://github.com/Shahriarnz14/SurvHTE-Bench .
Engineering Breakdown
Plain English
This paper introduces SurvHTE-Bench, the first standardized benchmark for evaluating heterogeneous treatment effect (HTE) estimation in survival analysis with censored data. The problem is critical for precision medicine and personalized policy—determining which treatments work best for which individuals when outcomes are right-censored (incomplete follow-up). The authors built a modular suite of synthetic and semi-synthetic datasets with known ground truth, systematically varying causal assumptions and survival dynamics, to enable reproducible evaluation of existing methods like Causal Survival Forests and survival meta-learners. This fills a major gap: prior work on HTE estimation lacked consistent evaluation practices, making it impossible to reliably compare methods in the survival setting.
Core Technical Contribution
The core contribution is SurvHTE-Bench itself—a comprehensive, modular benchmarking framework designed specifically for censored survival outcomes, which is structurally different from existing HTE benchmarks built for uncensored regression or classification. Rather than proposing a new algorithm, the authors engineered a reproducible evaluation infrastructure that systematically controls causal assumptions (confounding, unobserved confounders, instrumental variables) and survival dynamics (proportional hazards vs. non-proportional hazards, different censoring mechanisms). The benchmark includes synthetic data with ground-truth treatment effects, semi-synthetic variants that preserve real data distributions while injecting known effects, and standardized metrics for comparing methods across these scenarios. This is novel because survival analysis introduces identification challenges that uncensored settings don't face—the benchmark makes these challenges explicit and measurable.
How It Works
SurvHTE-Bench operates as a three-layer evaluation system: (1) synthetic data generation layer that creates survival datasets with known ground-truth HTEs by controlling the data-generating process—causal structure, treatment assignment mechanism, outcome model, and censoring distribution; (2) semi-synthetic layer that takes real survival datasets (e.g., from clinical trials or observational studies) and injects synthetic treatment effects while preserving the original covariate structure and censoring patterns; (3) evaluation layer that runs candidate HTE methods (Causal Survival Forests, survival meta-learners, outcome imputation approaches) on these datasets and compares predictions against ground truth using metrics like RMSE on treatment effects, concordance for ranking, and calibration measures. The key mechanism is that by controlling the data generation process, researchers can isolate how different causal assumptions, survival models, and censoring mechanisms affect method performance—you can test whether a method that assumes proportional hazards fails when hazards don't satisfy this assumption, or whether it degrades under heavy censoring.
Production Impact
For teams building precision medicine systems or personalized policy engines, SurvHTE-Bench provides a standardized way to select and validate HTE estimation methods before deployment, reducing the risk of choosing a method that fails on your specific data distribution. In production, you would use this benchmark to (1) choose between competing HTE algorithms by testing them on synthetic data that mimics your application's censoring rate and causal structure, (2) understand method behavior under distribution shift—e.g., how does your chosen estimator degrade if censoring increases from 20% to 50%?, and (3) validate the identification assumptions your method requires (does it assume no unmeasured confounding? what if you have instrumental variables?). The trade-off is that synthetic benchmarks may not capture all real-world pathologies—domain-specific validation is still required—and running multiple HTE methods during selection adds compute overhead, but this is typically a one-time cost. Integration is straightforward since the benchmark is modular: you can run specific scenarios rather than the full suite, making it practical for rapid iteration during model development.
Limitations and When Not to Use This
The benchmark is fundamentally limited to scenarios where the data-generating process can be fully specified and ground-truth effects are known; real clinical data has unmeasured confounding and other unknown mechanisms that synthetic data cannot capture, so excellent benchmark performance does not guarantee real-world validity. The paper does not address methods for handling time-varying treatments or complex causal graphs with feedback loops—it focuses on static treatment assignment at a single time point. Computational cost of running multiple methods across many benchmark scenarios is not thoroughly discussed, and it's unclear how practical this is for practitioners with limited resources. Additionally, the paper relies on semi-synthetic data for realism, but the quality of semi-synthetic benchmarks depends heavily on the original datasets chosen; if the original data have pathological properties or limited diversity, the benchmark's conclusions may not generalize.
Research Context
This work builds on a decade of research in causal forests and heterogeneous treatment effects (Athey, Wager, Kunzel, et al.), extending those ideas to the survival analysis setting where censoring fundamentally complicates identification and evaluation. Prior benchmarks for HTE estimation exist (e.g., IHDP, ACIC, Twins datasets) but were designed for uncensored outcomes and do not control for survival-specific challenges like varying censoring mechanisms or non-proportional hazards. The paper advances the parallel line of work on survival causal inference (Künzel et al. on survival forests, recent meta-learners for survival outcomes) by providing the first standardized evaluation infrastructure, analogous to how benchmarks like ImageNet or GLUE drove progress in computer vision and NLP. This opens a research direction toward improving HTE methods specifically for the survival setting—authors can now isolate which algorithmic components fail under specific causal or survival dynamics, enabling targeted method development.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::