Global Interpretability via Automated Preprocessing: A Framework Inspired by Psychiatric Questionnaires
| Authors | Eric V. Strobl |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2602.23459 |
| Download | |
| Categories | cs.LG, stat.ML |
Abstract
Psychiatric questionnaires are highly context sensitive and often only weakly predict subsequent symptom severity, which makes the prognostic relationship difficult to learn. Although flexible nonlinear models can improve predictive accuracy, their limited interpretability can erode clinical trust. In fields such as imaging and omics, investigators commonly address visit- and instrument-specific artifacts by extracting stable signal through preprocessing and then fitting an interpretable linear model. We adopt the same strategy for questionnaire data by decoupling preprocessing from prediction: we restrict nonlinear capacity to a baseline preprocessing module that estimates stable item values, and then learn a linear mapping from these stabilized baseline items to future severity. We refer to this two-stage method as REFINE (Redundancy-Exploiting Follow-up-Informed Nonlinear Enhancement), which concentrates nonlinearity in preprocessing while keeping the prognostic relationship transparently linear and therefore globally interpretable through a coefficient matrix, rather than through post hoc local attributions. In experiments, REFINE outperforms other interpretable approaches while preserving clear global attribution of prognostic factors across psychiatric and non-psychiatric longitudinal prediction tasks.
Engineering Breakdown
Plain English
This paper addresses a critical problem in clinical AI: psychiatric questionnaires are weakly predictive of future symptoms and hard for doctors to trust because the models explaining them are black boxes. The authors propose a two-stage approach inspired by medical imaging pipelines: first, use a flexible nonlinear model to preprocess noisy questionnaire responses into stable, context-corrected item values; second, fit a simple linear model from these preprocessed items to future symptom severity. This decoupling lets the system achieve better accuracy than nonlinear-only baselines while maintaining the interpretability of a linear model that clinicians can understand and validate.
Core Technical Contribution
The key insight is decoupling nonlinear preprocessing from linear prediction, deliberately restricting nonlinear capacity to artifact removal rather than end-to-end prediction. Prior work either accepted poor accuracy with linear models or traded interpretability for better nonlinear predictors; this paper shows you can have both by borrowing the preprocessing-then-interpret paradigm from imaging and genomics fields. The technical novelty lies in formalizing questionnaire data artifacts (context sensitivity, instrument drift) as a preprocessing problem solvable by a compact nonlinear encoder, leaving the learned relationship to outcome as a transparent linear function. This is architecturally simple but conceptually important because it identifies a structural constraint that works specifically for clinical questionnaire data.
How It Works
The system has two sequential stages. First, a nonlinear baseline preprocessing module takes raw questionnaire responses (which are noisy, context-dependent, and instrument-specific) and learns to output stabilized item values—essentially a learned normalization that removes systematic artifacts. This module has finite capacity and is not jointly optimized with prediction, keeping it focused on denoising rather than predicting outcomes. Second, a linear regression layer maps these stabilized items directly to future symptom severity scores. During training, the nonlinear preprocessor is trained to maximize interpretability of the downstream linear model (e.g., via a regularization term on the linear weights), while the linear layer is trained via standard regression loss on held-out clinical outcomes. At inference, a new patient's questionnaire responses flow through the preprocessor to get stable items, then through the linear model to get a severity prediction plus transparent per-item coefficients a clinician can inspect.
Production Impact
In practice, this approach solves a real credibility problem: clinicians often reject black-box psychiatric severity predictions because they cannot verify the logic against their domain knowledge. By guaranteeing a linear readout, this method produces models where each item's contribution to a severity forecast is explicit and auditable. The production pipeline becomes simpler too—no need to maintain complex feature engineering or model compression; the nonlinear stage is trained once and then frozen, and the linear stage can be retrained quickly if new outcome data arrives. Compute cost is minimal: preprocessing is a single forward pass through a small neural network, and prediction is a dot product. Latency is sub-millisecond. The main trade-off is that the approach assumes the questionnaire artifacts are learnable by a deterministic nonlinear function; if outcomes depend on complex interactions between raw items that the preprocessor averages away, accuracy will suffer. Clinical deployment requires rigorous validation that the linear coefficients are stable across patient subgroups and that the model's confidence intervals are well-calibrated.
Limitations and When Not to Use This
The paper does not address scenarios where the true prognostic relationship is fundamentally nonlinear—for instance, if severe depression manifests only at extreme combinations of item responses that a linear model cannot capture. It assumes questionnaire artifacts are stationary and learnable from historical data; in real clinical settings, new instruments, changed administration protocols, or demographic shifts could invalidate the learned preprocessing. The approach also requires a substantial labeled dataset to train both the preprocessor and the linear layer reliably, which may be scarce in specialized psychiatric sub-populations. Finally, the paper does not deeply explore the trade-off between preprocessing capacity and downstream interpretability—too weak a preprocessor may leave clinically important signal mixed into noise, while too strong a preprocessor may overfit and produce unstable item estimates that break the linear model's generalization.
Research Context
This work sits at the intersection of interpretable machine learning and clinical AI, inspired by established pipelines in medical imaging and genomics where preprocessing (e.g., image normalization, batch correction) decouples from statistical modeling. It responds to growing skepticism of end-to-end deep learning in healthcare, where regulatory and ethical pressures demand human-auditable decision paths. The paper builds on prior work in enforced model transparency (linear models, additive models) and self-supervised representation learning for denoising, but applies these ideas to a domain—psychiatric assessment—where weak signal-to-noise ratio and context sensitivity are the defining challenge. It opens a research direction: can this preprocessing-then-interpret paradigm be applied to other weak-signal clinical domains (e.g., brief neuropsychological batteries, symptom checklists), and how do we optimally trade off preprocessor complexity against interpretability?
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::