ProtoFlow: Mitigating Forgetting in Class-Incremental Remote Sensing Segmentation via Low-Curvature Prototype Flow
| Authors | Jiekai Wu et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2604.03212 |
| Download | |
| Categories | cs.CV |
Abstract
Remote sensing segmentation in real deployment is inherently continual: new semantic categories emerge, and acquisition conditions shift across seasons, cities, and sensors. Despite recent progress, many incremental approaches still treat training steps as isolated updates, which leaves representation drift and forgetting insufficiently controlled. We present ProtoFlow, a time-aware prototype dynamics framework that models class prototypes as trajectories and learns their evolution with an explicit temporal vector field. By jointly enforcing low-curvature motion and inter-class separation, ProtoFlow stabilizes prototype geometry throughout incremental learning. Experiments on standard class- and domain-incremental remote sensing benchmarks show consistent gains over strong baselines, including up to 1.5-2.0 points improvement in mIoUall, together with reduced forgetting. These results suggest that explicitly modeling temporal prototype evolution is a practical and interpretable strategy for robust continual remote sensing segmentation.
Engineering Breakdown
Plain English
ProtoFlow addresses a critical problem in remote sensing: deployed systems must continuously learn new object categories and adapt to changing environmental conditions (different seasons, cities, sensors) without forgetting previously learned classes. The paper proposes modeling class prototypes as temporal trajectories governed by a learned vector field, which keeps prototype representations stable while learning incrementally. Experiments on class-incremental and domain-incremental benchmarks show consistent improvements over existing approaches by explicitly controlling representation drift and inter-class separation throughout the learning process.
Core Technical Contribution
The core novelty is treating class prototypes as dynamic objects with explicit temporal evolution rather than static points. Instead of updating prototypes independently at each training step, ProtoFlow learns a vector field that governs how prototypes should move through representation space over time, with joint constraints: (1) low-curvature motion to prevent erratic prototype jumps, and (2) inter-class separation to maintain discriminability. This temporal modeling framework is fundamentally different from prior incremental learning approaches that treat each learning stage as an isolated update without considering the geometric stability of the learned representation.
How It Works
ProtoFlow operates by maintaining per-class prototype vectors that represent learned feature distributions. At each incremental learning stage, instead of directly updating prototypes with new data, the method learns a smooth vector field that describes how all prototypes should evolve. The input is new class samples at each stage; the system computes prototype movements guided by: (1) a curvature regularization term that penalizes sharp direction changes in prototype trajectories (low-curvature constraint), and (2) a margin-based loss that pushes different class prototypes apart in feature space. The temporal vector field is learned end-to-end alongside the feature encoder, so prototype updates respect both past geometry and future discriminability. Output is a sequence of stable prototype configurations across all incremental steps that minimizes both representation drift and catastrophic forgetting.
Production Impact
For engineers deploying remote sensing systems, ProtoFlow solves the practical problem of degrading accuracy over time as new object categories are added and sensor conditions change. Rather than retraining from scratch (expensive, memory-intensive) or naive fine-tuning (causes forgetting), this approach provides a principled way to absorb new classes while maintaining performance on old ones. The temporal modeling adds computational overhead during training—you must learn and store a vector field in addition to the feature encoder—but inference remains lightweight (just prototype matching). Integration is straightforward: replace static prototype updates with vector-field-guided updates in any prototype-based incremental learning pipeline. The main trade-off is increased training complexity and memory for vector field storage versus significant accuracy gains across both new and old tasks.
Limitations and When Not to Use This
ProtoFlow assumes that smooth prototype evolution is always beneficial, which may not hold if the underlying task structure changes drastically (e.g., entirely new semantic domains). The method requires reliable estimates of prototype trajectories, which becomes challenging with very few samples per new class or highly imbalanced incremental scenarios. The paper's evaluation appears limited to remote sensing; generalization to other domains (medical imaging, autonomous driving) is not demonstrated. Additionally, the approach still requires access to some representation of old classes or old data for maintaining inter-class separation—full class-incremental learning without any old-class data (a harder setting) is not fully addressed.
Research Context
This work builds on a growing body of research in continual/incremental learning for vision, extending beyond standard class-incremental learning to handle realistic domain shifts across sensors and seasons. ProtoFlow improves over prototype-based incremental methods (e.g., DER, PODNet) by adding temporal structure, aligning with recent trends in trajectory-based learning and neural ODEs. The paper tackles domain-incremental remote sensing—a practical but understudied benchmark where test conditions differ from training—opening research directions toward multi-modal incremental learning. This contribution is timely as remote sensing systems increasingly deploy with streaming data acquisition and evolving labeling requirements.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::