CLoPA: Continual Low Parameter Adaptation of Interactive Segmentation for Medical Image Annotation
| Authors | Parhom Esmaeili et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2603.06426 |
| Download | |
| Categories | cs.CV, cs.AI, cs.LG |
Abstract
Interactive segmentation enables clinicians to guide annotation, but existing zero-shot models like nnInteractive fail to consistently reach expert-level performance across diverse medical imaging tasks. Because annotation campaigns produce a growing stream of task-specific labelled data, online adaptation of the segmentation model is a natural complement to zero-shot inference. We propose CLoPA, a continual adaptation strategy that tunes a small fraction of nnInteractive's parameters on the annotation cache, triggered by lightweight episode scheduling. CLoPA requires no new parameters or changes to the inference pipeline, and operates entirely within the existing annotation workflow. Across eight Medical Segmentation Decathlon tasks spanning diverse anatomical targets and imaging characteristics, CLoPA rapidly elevates performance to expert-level, even for tasks where nnInteractive previously failed, with the majority of gains realised after a single training episode. We show that the benefits of tuning different parameter groups depends on task characteristics and data regimes. Also, that for targets with complex geometries (e.g., hepatic vessels), instance normalisation and low-level feature tuning saturates, suggesting a need for deeper feature-representation alignment in the most challenging scenarios.
Engineering Breakdown
Plain English
This paper addresses a real clinical problem: zero-shot medical image segmentation models like nnInteractive work out-of-the-box but often miss expert-level accuracy on diverse anatomy and imaging modalities. The authors propose CLoPA, a continual adaptation strategy that incrementally updates a frozen model's parameters using annotation data generated during actual clinical workflows—essentially learning from human corrections in real-time. CLoPA is lightweight, requires no architectural changes, and operates seamlessly within existing annotation pipelines. Testing across eight Medical Segmentation Decathlon tasks shows this approach can close the gap between zero-shot and expert performance without requiring expensive retraining cycles.
Core Technical Contribution
The core innovation is a parameter-efficient continual learning framework that adapts pre-trained medical segmentation models to task-specific data streams without architectural modification. Unlike standard fine-tuning which requires full retraining or separate model variants, CLoPA selectively tunes only a small fraction of parameters in nnInteractive triggered by a lightweight episode scheduler that decides when adaptation is necessary. This is fundamentally different from prior approaches because it operates entirely within the annotation workflow itself—clinicians naturally generate labeled data during segmentation, and CLoPA converts this into online model improvement without disrupting clinical processes. The method sidesteps the usual trade-off between zero-shot generalization and task-specific performance by making adaptation incremental, triggered, and parameter-efficient.
How It Works
CLoPA takes a pre-trained zero-shot segmentation model (nnInteractive) and wraps it with an episode scheduling mechanism that monitors annotation quality. When the scheduler detects that the model's performance is below a threshold on incoming annotations, it triggers a lightweight adaptation phase where only a carefully selected subset of model parameters are updated using the annotation cache—recent labeled examples from the current segmentation task. The input is the frozen base model weights plus an accumulating buffer of (image, ground-truth mask) pairs from ongoing clinical annotations. During adaptation episodes, these cached examples are used to compute gradients and update the selected parameters via standard optimization (likely SGD or Adam with small learning rates to avoid catastrophic forgetting). The updated model then continues inference on new incoming cases, and the process repeats asynchronously—clinicians never see training loops, they just see improved segmentation suggestions over time as the cache grows.
Production Impact
For clinical deployment, CLoPA solves the critical problem of model performance drift without requiring data science team intervention or expensive retraining pipelines. Instead of shipping a single frozen model that degrades on new anatomy or imaging protocols, teams can deploy one base model and let it quietly adapt to local data distributions via clinician annotations—effectively getting personalization and domain adaptation for free during normal annotation workflows. The production pipeline remains unchanged: inference latency and memory footprint are identical to the base model since parameter updates happen offline and only infrequently. Trade-offs are minimal: you need to maintain an annotation cache (modest storage cost), monitor scheduling criteria (lightweight CPU compute), and handle parameter update synchronization, but these fit naturally into existing DICOM/clinical IT infrastructure. The main risk is ensuring adaptation doesn't introduce patient-safety regressions, so you'd want validation gates and rollback mechanisms before deploying adapted checkpoints to new clinicians.
Limitations and When Not to Use This
The paper does not address how to select which parameters to adapt without degenerating performance on the original zero-shot task—catastrophic forgetting is mentioned implicitly but the parameter selection strategy isn't clearly specified. CLoPA assumes annotation data arrives as a continuous stream with meaningful label quality, but in practice annotation errors, inconsistent raters, and sparse feedback could poison the cache and cause harmful adaptations. The approach hasn't been tested on truly out-of-distribution scenarios (e.g., entirely new imaging modalities or anatomy the base model has never seen) where online adaptation may be too slow or insufficient. Finally, the eight Medical Segmentation Decathlon tasks tested are well-curated benchmarks; real-world clinical deployment would face messier data, smaller annotation budgets per task, and stricter regulatory constraints on model drift that aren't addressed in the current work.
Research Context
This work builds directly on nnInteractive, a zero-shot interactive segmentation framework, and sits at the intersection of continual learning and few-shot medical imaging. It extends prior work on parameter-efficient adaptation (conceptually related to LoRA and adapter modules in NLP) into the medical imaging domain where you have human-in-the-loop workflows. The broader context is the Medical Segmentation Decathlon benchmark, which established the challenge of generalizing across diverse anatomical targets—CLoPA shows that zero-shot models can be made practical by allowing lightweight task-specific refinement. This opens research directions in online learning for clinical AI, curriculum-based scheduling for when adaptation is beneficial, and safety-aware continual learning where you need guarantees that adaptation doesn't harm previous performance.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::