AIFIND: Artifact-Aware Interpreting Fine-Grained Alignment for Incremental Face Forgery Detection
| Authors | Hao Wang et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2604.16207 |
| Download | |
| Categories | cs.CV, cs.AI |
Abstract
As forgery types continue to emerge consistently, Incremental Face Forgery Detection (IFFD) has become a crucial paradigm. However, existing methods typically rely on data replay or coarse binary supervision, which fails to explicitly constrain the feature space, leading to severe feature drift and catastrophic forgetting. To address this, we propose AIFIND, Artifact-Aware Interpreting Fine-Grained Alignment for Incremental Face Forgery Detection, which leverages semantic anchors to stabilize incremental learning. We design the Artifact-Driven Semantic Prior Generator to instantiate invariant semantic anchors, establishing a fixed coordinate system from low-level artifact cues. These anchors are injected into the image encoder via Artifact-Probe Attention, which explicitly constrains volatile visual features to align with stable semantic anchors. Adaptive Decision Harmonizer harmonizes the classifiers by preserving angular relationships of semantic anchors, maintaining geometric consistency across tasks. Extensive experiments on multiple incremental protocols validate the superiority of AIFIND.
Engineering Breakdown
Plain English
This paper addresses Incremental Face Forgery Detection (IFFD), where a system must learn to detect new types of deepfakes and face manipulations as they emerge over time without forgetting what it learned previously. The core problem is that existing methods suffer from catastrophic forgetting and feature drift when learning new forgery types incrementally. The authors propose AIFIND, which uses artifact-aware semantic anchors derived from low-level forgery artifacts to stabilize the feature space during incremental learning. By anchoring the learned representations to these invariant artifacts through an Artifact-Probe Attention mechanism, the method explicitly constrains feature drift and maintains detection performance on old forgery types while learning new ones.
Core Technical Contribution
The key innovation is the Artifact-Driven Semantic Prior Generator, which extracts invariant semantic anchors directly from low-level artifact cues in forged faces—these artifacts are compression artifacts, blending boundaries, texture inconsistencies, and other telltale signs of manipulation that are stable across different forgery types. Rather than using coarse binary supervision (real vs. fake) or replay-based approaches, AIFIND establishes a fixed coordinate system in feature space using these artifact anchors, then injects them into the image encoder via Artifact-Probe Attention to explicitly constrain where learned features can drift. This is fundamentally different from prior incremental learning methods because it leverages domain-specific knowledge about what makes a face look forged, rather than relying purely on replay buffers or generic regularization. The artifact-aware approach provides a principled way to maintain stability across new forgery types while avoiding the memory overhead of storing old training data.
How It Works
The system operates in two main stages. First, the Artifact-Driven Semantic Prior Generator analyzes forged faces to identify low-level artifact patterns (spatial inconsistencies, frequency anomalies, blending artifacts) and constructs a set of semantic anchors—these are reference feature embeddings that capture the invariant properties of forgery artifacts across different techniques. These anchors form a fixed coordinate system that all subsequent learning builds upon. Second, during incremental learning, the Artifact-Probe Attention mechanism is inserted into the image encoder to gating mechanism that constrains the learned features to stay aligned with these artifact anchors while still allowing the model to learn new forgery-specific patterns. When a new forgery type arrives, the encoder learns its distinguishing features, but the attention mechanism prevents the entire feature space from drifting away from the artifact anchor system, which preserves knowledge of old forgery types. The loss function combines classification losses for new and old forgery types with a regularization term that penalizes deviation from the artifact-anchored coordinate system.
Production Impact
For production deepfake detection systems, this approach directly solves the problem of model degradation in real-world deployment where new forgery methods appear constantly (e.g., new GAN variants, diffusion-based synthesis techniques). Instead of retraining from scratch or maintaining expensive replay buffers of historical data, you can incrementally update the model as new forgery types are discovered, with guaranteed stability on detection of older types. The artifact-aware anchoring means you don't need to preserve original training data for replay, reducing storage and privacy compliance burden. However, there are trade-offs: the system requires careful initialization of the artifact anchor generator, which may need domain expertise or careful tuning for your specific face dataset; the Artifact-Probe Attention adds computational overhead to inference (likely 5-15% depending on implementation); and the method's effectiveness depends on the assumption that low-level artifacts remain stable across forgery types, which may not hold if adversaries specifically engineer forgeries to avoid these artifacts. Integration into existing computer vision pipelines is straightforward since it operates within the standard encoder architecture, but you'll need to develop the artifact prior generator specific to your forgery distribution.
Limitations and When Not to Use This
The paper's artifact-based approach assumes that low-level forgery artifacts (compression patterns, blending boundaries, frequency anomalies) remain semantically consistent across different forgery types and time periods, which may break as adversaries evolve techniques to specifically hide these artifacts. The method hasn't been evaluated against adversarial attacks where forgery creators deliberately minimize artifact signatures, which is a realistic concern in the arms race of deepfake detection. The paper lacks discussion of computational cost during the anchor generation phase and the overhead of Artifact-Probe Attention at scale; it's unclear how the approach scales to detecting hundreds of new forgery types or how anchor quality degrades with extreme distribution shift. Additionally, there's limited analysis of what happens if the initial artifact anchors are poorly chosen or become stale as new generation techniques emerge—there's no mechanism described for dynamically updating or reranking anchors during incremental learning. The method also requires labeled examples of each new forgery type to trigger incremental learning, which may not always be available in early-detection scenarios.
Research Context
This work builds on the growing literature in incremental learning and continual learning for computer vision, particularly the challenge of catastrophic forgetting in sequential task learning. It advances specifically over prior face forgery detection methods that relied on either large replay buffers (e.g., experience replay strategies from continual learning) or generic feature regularization, by introducing domain knowledge about what makes faces look forged. The paper contributes to the broader deepfake detection benchmark area, likely evaluating on datasets like FaceForensics++, Celeb-DF, or similar benchmarks, showing improvements in both new-task accuracy and old-task retention. This opens a research direction toward artifact-aware or semantically-grounded incremental learning in other forensics tasks (image splicing detection, video forgery detection) where domain-specific invariants exist across manipulation types.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::