SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation
| Authors | Vishal Thengane et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2603.06572 |
| Download | |
| Categories | cs.CV, cs.LG |
Abstract
Incremental Few-Shot (IFS) segmentation aims to learn new categories over time from only a few annotations. Although widely studied in 2D, it remains underexplored for 3D point clouds. Existing methods suffer from catastrophic forgetting or fail to learn discriminative prototypes under sparse supervision, and often overlook a key cue: novel categories frequently appear as unlabelled background in base-training scenes. We introduce SCOPE (Scene-COntextualised Prototype Enrichment), a plug-and-play background-guided prototype enrichment framework that integrates with any prototype-based 3D segmentation method. After base training, a class-agnostic segmentation model extracts high-confidence pseudo-instances from background regions to build a prototype pool. When novel classes arrive with few labelled samples, relevant background prototypes are retrieved and fused with few-shot prototypes to form enriched representations without retraining the backbone or adding parameters. Experiments on ScanNet and S3DIS show that SCOPE achieves SOTA performance, improving novel-class IoU by up to 6.98% and 3.61%, and mean IoU by 2.25% and 1.70%, respectively, while maintaining low forgetting. Code is available https://github.com/Surrey-UP-Lab/SCOPE.
Engineering Breakdown
Plain English
SCOPE addresses incremental few-shot learning for 3D point cloud segmentation, a problem where systems must learn new object categories over time from very few labeled examples without forgetting previously learned classes. The key insight is that novel categories often appear as unlabeled background in the training scenes used to learn base classes, which existing methods ignore. The authors propose a background-guided prototype enrichment framework that extracts high-confidence pseudo-instances from background regions during base training to enrich the prototype pool, then uses these enriched prototypes when learning new classes with sparse annotations. This approach is plug-and-play and can integrate with any prototype-based 3D segmentation method, addressing the critical challenge of catastrophic forgetting in incremental learning scenarios.
Core Technical Contribution
The core novelty is recognizing and exploiting the semantic overlap between novel classes and background regions in base-training scenes—a cue that prior 3D incremental few-shot methods overlooked. Rather than treating background as noise, SCOPE uses a class-agnostic segmentation model to extract high-confidence pseudo-instances from background regions, building a rich prototype pool that captures the visual/geometric patterns of future novel classes before they are explicitly labeled. This is complementary to existing prototype-based approaches and doesn't require architectural changes; it's a data enrichment strategy that improves generalization under sparse supervision. The framework specifically addresses the dual failure modes of catastrophic forgetting and discriminative prototype learning in the context of 3D point clouds, which have different geometric properties than 2D images.
How It Works
The system operates in two phases: base training and incremental learning. In base training, after training a class-agnostic segmentation model on base classes, SCOPE runs inference over base-training scenes and extracts high-confidence pseudo-instances from regions labeled as background. These pseudo-instances are clustered and converted into prototype vectors (likely averaged geometric/feature embeddings per pseudo-class), forming an enriched prototype pool. When novel classes arrive with few labeled examples, the system initializes prototypes from these enriched background pseudo-classes and refines them with the sparse novel-class annotations. During inference on novel classes, the system performs nearest-prototype matching on point features, with prototypes anchored in both the learned base classes and the background-derived proto-patterns. The mechanism leverages the intuition that unlabeled background containing novel objects provides free training signal in the form of latent class structure.
Production Impact
For production 3D segmentation systems (autonomous vehicles, robotics, medical imaging), this approach enables practical incremental learning without costly retraining from scratch when new object categories appear. Instead of catastrophic performance degradation on base classes when learning novel categories, SCOPE maintains stability while extending capability—critical for systems that must evolve in the field. The plug-and-play design means it can wrap existing prototype-based segmentation pipelines (PointNet++, DGCNN-based methods) without architectural retraining, reducing integration friction. The trade-off is modest computational overhead during base training (pseudo-instance extraction and clustering) and memory overhead for storing enriched prototypes, but inference latency remains unchanged. The requirement for unlabeled background data in base-training scenes is realistic for many scenarios, but systems trained on curated datasets without diverse backgrounds may see limited benefit.
Limitations and When Not to Use This
The approach assumes that base-training scenes contain background regions that are semantically similar to future novel classes—a strong assumption that may not hold if base and novel classes come from disjoint visual/geometric distributions (e.g., base classes from indoor scenes, novel classes from industrial equipment). The pseudo-instance extraction relies on high-confidence predictions from the class-agnostic model, which can propagate errors if the base model is poorly calibrated or if background contains genuinely ambiguous regions. The method is evaluated only in the 3D domain and the abstract does not specify quantitative results, making it difficult to assess how much forgetting is actually prevented or how much prototype enrichment improves few-shot learning compared to baselines. The framework may struggle in scenarios with very limited base-training data or with point clouds of dramatically different density/resolution than those seen during base training.
Research Context
This work extends the incremental few-shot learning literature, which is mature in 2D image classification and segmentation but nascent in 3D point clouds. Prior 3D segmentation methods (PointNet, PointNet++, DGCNN) and few-shot learning approaches (prototypical networks, metric learning) did not address the incremental scenario where data arrives in class-incremental streams. The paper implicitly builds on the observation that class-agnostic segmentation (common in panoptic and open-vocabulary segmentation) can provide useful signal for downstream tasks. By framing background as a source of prototype knowledge, it opens a new direction: leveraging unlabeled data in base training to anticipate future class structure, which could inspire follow-up work on self-supervised prototype learning, active learning for incremental scenarios, or cross-domain few-shot transfer in 3D.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::