ArtifactNet: Detecting AI-Generated Music via Forensic Residual Physics
| Authors | Heewon Oh |
| Year | 2026 |
| HF Upvotes | 0 |
| arXiv | 2604.16254 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
We present ArtifactNet, a lightweight framework that detects AI-generated music by reframing the problem as forensic physics -- extracting and analyzing the physical artifacts that neural audio codecs inevitably imprint on generated audio. A bounded-mask UNet (ArtifactUNet, 3.6M parameters) extracts codec residuals from magnitude spectrograms, which are then decomposed via HPSS into 7-channel forensic features for classification by a compact CNN (0.4M parameters; 4.0M total). We introduce ArtifactBench, a multi-generator evaluation benchmark comprising 6,183 tracks (4,383 AI from 22 generators and 1,800 real from 6 diverse sources). Each track is tagged with bench_origin for fair zero-shot evaluation. On the unseen test partition (n=2,263), ArtifactNet achieves F1 = 0.9829 with FPR = 1.49%, compared to CLAM (F1 = 0.7576, FPR = 69.26%) and SpecTTTra (F1 = 0.7713, FPR = 19.43%) evaluated under identical conditions with published checkpoints. Codec-aware training (4-way WAV/MP3/AAC/Opus augmentation) further reduces cross-codec probability drift by 83% (Delta = 0.95 -> 0.16), resolving the primary codec-invariance failure mode. These results establish forensic physics -- direct extraction of codec-level artifacts -- as a more generalizable and parameter-efficient paradigm for AI music detection than representation learning, using 49x fewer parameters than CLAM and 4.8x fewer than SpecTTTra.
Engineering Breakdown
Plain English
ArtifactNet is a lightweight system for detecting AI-generated music by analyzing the physical artifacts that neural audio codecs leave behind during generation. The approach uses a small UNet model (3.6M parameters) to extract codec residuals from spectrograms, then decomposes these signals into 7-channel forensic features using harmonic-percussive source separation (HPSS) for final classification. On a new benchmark of 6,183 tracks (4,383 AI-generated from 22 different generators and 1,800 real recordings), the system achieved 98.29% F1-score with only 1.49% false positive rate on unseen test data, demonstrating strong generalization across different music generators.
Core Technical Contribution
The core novelty is reframing AI music detection as a forensic physics problem rather than treating it as a black-box classification task. Instead of learning direct discriminative features between real and generated audio, the authors identified that neural codecs (used in diffusion models and other generative systems) imprint consistent, measurable physical artifacts in the frequency domain that can be extracted and analyzed systematically. The bounded-mask UNet architecture specifically targets these residual artifacts while keeping model size minimal (3.6M parameters), and the use of HPSS decomposition to create a structured 7-channel feature representation provides interpretability — you can see exactly which frequency and source components expose the generation process. This physics-grounded approach is fundamentally different from prior end-to-end discriminators, as it leverages domain knowledge about how audio generation systems actually work.
How It Works
The input is an audio waveform that gets converted to a magnitude spectrogram. The bounded-mask UNet then processes this spectrogram in a constrained way — the 'bounded mask' likely means the model learns to attend only to frequency regions where codec artifacts are likely present, rather than analyzing the entire spectrum. This produces a residual map showing where codec artifacts are concentrated. The residual output is then fed into HPSS (harmonic-percussive source separation), a classical signal processing technique that decomposes audio into 7 different components: harmonic instruments, percussive elements, and their various harmonic/percussive combinations. These 7 channels become the forensic feature representation, capturing different aspects of how the codec-induced artifacts manifest across different sound types. Finally, a compact CNN classifier (0.4M parameters) takes these 7-channel features and makes a binary real-vs-generated decision. The full pipeline is only 4.0M parameters total, making it efficient for deployment while maintaining 98.29% F1-score.
Production Impact
For music platforms and streaming services, this enables reliable detection of AI-generated content at scale without prohibitive compute costs — the full model is under 4MB in memory, allowing inference on edge devices or lightweight servers. The 1.49% false positive rate is critical for production: it means legitimate human artists won't be accidentally flagged as AI-generated at a rate that causes customer friction. The zero-shot generalization (trained on one set of generators, tested on different unseen generators from the ArtifactBench dataset) suggests the approach works across different AI music generation architectures, not just the ones in the training set, reducing the need for constant retraining as new generators emerge. However, the production pipeline requires spectrogram extraction and HPSS decomposition before the model runs, adding latency and complexity compared to end-to-end models — you'd need to integrate signal processing libraries and manage the multi-channel feature computation. Integration would involve establishing confidence thresholds (0.5 default classification, but adjustable based on false positive vs. false negative trade-offs for your use case) and handling the 1-2% failure cases with secondary human review or additional detector ensemble.
Limitations and When Not to Use This
The paper evaluates only on music generated by 22 specific AI generators and doesn't detail how well the approach generalizes to fundamentally different audio codec architectures or newer generative models released after the benchmark was built — as new codec designs emerge, the 'physics' of artifacts may shift. The 1.49% false positive rate, while low, still means ~15-20 legitimate artists get flagged per 1,000 uploads on a typical platform, which could create support burden and user trust issues. The approach assumes access to raw waveforms or clean spectrograms; compressed formats (MP3, AAC) or heavily processed/mixed audio may degrade the artifact signal, as codec compression itself introduces physical distortions that could confound the forensic analysis. The paper doesn't thoroughly discuss adversarial robustness — whether someone could intentionally add benign artifacts to real music to fool the detector, or whether carefully crafted audio post-processing could strip codec artifacts from AI-generated music before uploading.
Research Context
This work sits at the intersection of audio forensics and generative model detection, building on prior research in music source separation (HPSS is a classical technique from the 2010s) and the emerging field of detecting synthetic media. It directly addresses a gap in AI safety research: while image deepfake detection matured significantly, audio generation detection remained underdeveloped, especially for music. The introduction of ArtifactBench with 6,183 tracks and careful tagging of generator provenance provides a new standard evaluation dataset for the field, similar to how ImageNet and later deepfake benchmarks shaped computer vision. The physics-grounded approach contrasts with concurrent work on end-to-end discriminators and opens a research direction toward interpretable detection mechanisms — future work could investigate whether similar forensic approaches work for other audio (speech, podcasts) or entirely different modalities (video codec artifacts, image generation traces).
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::