AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images
| Authors | Bo Zhang et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2604.28177 |
| Download | |
| Categories | cs.CV, cs.CY |
Abstract
We introduce AEGIS, A holistic benchmark for Evaluating forensic analysis of AI-Generated academic ImageS. Compared to existing benchmarks, AEGIS features three key advances: (1) Domain-Specific Complexity: covering seven academic categories with 39 fine-grained subtypes, exposing intrinsic forensic difficulty, where even GPT-5.1 reaches 48.80% overall performance and expert models achieve only limited localization accuracy (IoU 30.09%); (2) Diverse Forgery Simulations: modeling four prevalent academic forgery strategies across 25 generative models, with 11 yielding average forensic accuracy below 50%, showing that forensics lag behind generative advances; and (3) Multi-Dimensional Forensic Evaluation: jointly assessing detection, reasoning, and localization, revealing complementary strengths between model families, with multimodal large language models (MLLMs) at 84.74% accuracy in textual artifact recognition and expert detectors peaking at 79.54% accuracy in binary authenticity detection. By evaluating 25 leading MLLMs, nine expert models, and one unified multimodal understanding and generation model, AEGIS serves as a diagnostic testbed exposing fundamental limitations in academic image forensics.
Engineering Breakdown
Plain English
AEGIS is a comprehensive benchmark for detecting and analyzing fake images generated by AI models in academic contexts. The benchmark covers seven academic disciplines with 39 specific image subtypes, tests 25 different generative models including 11 that fool detection systems over 50% of the time, and evaluates three forensic capabilities: detecting forgeries, explaining why they're fake, and pinpointing where the manipulation occurred. Even the most advanced models like GPT-5.1 only achieve 48.8% accuracy on overall detection, while localization (identifying exactly where the fake part is) reaches just 30.09% intersection-over-union—showing that AI-generated image forensics is still far behind generative model capabilities.
Core Technical Contribution
The paper's main innovation is constructing a large-scale, domain-specific forensic evaluation benchmark that exposes gaps between generative model advances and detection capabilities. Unlike previous benchmarks that use generic images, AEGIS focuses specifically on academic content (charts, diagrams, microscopy images, etc.) where forgeries are particularly dangerous and reveal domain-specific forensic challenges. The authors modeled four distinct academic forgery strategies (insertion, removal, modification, synthesis) across 25 generative models, creating a quantitative framework showing that 11 of these models produce images that evade detection more than half the time. The three-way evaluation (detection + reasoning + localization) reveals that these tasks require complementary model capabilities and aren't simply interdependent.
How It Works
AEGIS starts with real academic images across seven domains (e.g., microscopy, X-rays, satellite imagery, charts) and creates synthetic forgeries using four strategies: inserting fabricated elements (adding fake data points), removing authentic elements (deleting real features), modifying existing content (tweaking measurements), and full synthesis (generating completely fake images). For each strategy, the authors run the forgery through 25 generative models to create realistic academic fakes. The benchmark then passes these images to detection models which perform three tasks in sequence: (1) binary classification (real or fake), (2) free-form reasoning about why they classified it that way, and (3) pixel-level segmentation to localize the forged region. The evaluation metrics track accuracy across all three dimensions, revealing which models are good detectors but poor at localization, or vice versa.
Production Impact
For production AI systems handling academic or scientific content (research databases, preprint servers, grant proposal systems), AEGIS provides a realistic benchmark to stress-test your forensic pipeline before deployment. If you're building a system to flag potentially fabricated scientific figures, this benchmark shows you'll need multiple specialized detectors: a binary classifier will catch roughly 48% of fakes, but you'll also need separate reasoning and localization models because detection accuracy alone doesn't guarantee you can explain or pinpoint the forgery to reviewers. The finding that 44% of tested generative models exceed 50% evasion rates means your detection model needs continuous retraining—these models rapidly become obsolete as new generation techniques emerge. Trade-offs include needing labeled training data from all seven academic domains (creating significant annotation overhead), running three separate inference passes per image (3x latency cost), and accepting that even state-of-the-art systems will miss roughly half of sophisticated academic forgeries.
Limitations and When Not to Use This
AEGIS evaluates only academic images in seven specific domains, so performance may not transfer to medical imaging, legal documents, or journalistic photos where different forensic artifacts appear. The benchmark tests against models available at publication time; newer architectures that emerge after 2026 may exploit undetected vulnerabilities not captured by the 25 models tested. The paper doesn't address adversarial attacks where attackers specifically engineer forgeries to fool AEGIS-trained detectors, nor does it model real-world deployment scenarios where images arrive as compressed JPEGs or with metadata stripped—the forensic difficulty likely increases significantly. Finally, the reasoning evaluation component is not fully detailed in the abstract, leaving unclear whether this uses LLM-based explanation or structured reasoning, and how subjective vs. objective this evaluation actually is.
Research Context
This work builds on a decade of AI-forensics research (detecting deepfakes, GAN-generated images) but is the first to focus specifically on academic domain forgeries, which present unique challenges because academic images have domain-specific visual patterns that generative models can now replicate convincingly. Prior benchmarks like FaceForensics++ and NIST-led deepfake detection challenges operated on general imagery; AEGIS creates the academic equivalent and shows that domain-specific complexity significantly increases forensic difficulty. The paper advances the evaluation framework itself by moving beyond binary detection (real/fake) to multi-dimensional assessment (detection + reasoning + localization), a methodology now becoming standard as forensic systems need to provide auditable evidence. This opens a research direction toward interpretable forensics in specialized domains like medicine, materials science, and physics, where stakeholders demand not just detection but verifiable proof of manipulation.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::