Enhancing Authorship Attribution with Synthetic Paintings
| Authors | Clarissa Loures et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2603.04343 |
| Download | |
| Categories | cs.CV, cs.LG |
Abstract
Attributing authorship to paintings is a historically complex task, and one of its main challenges is the limited availability of real artworks for training computational models. This study investigates whether synthetic images, generated through DreamBooth fine-tuning of Stable Diffusion, can improve the performance of classification models in this context. We propose a hybrid approach that combines real and synthetic data to enhance model accuracy and generalization across similar artistic styles. Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings. By integrating generative and discriminative methods, this work contributes to the development of computer vision techniques for artwork authentication in data-scarce scenarios.
Engineering Breakdown
Plain English
This paper addresses the problem of training painting authorship attribution models when real artwork datasets are scarce. The authors use DreamBooth fine-tuning of Stable Diffusion to generate synthetic paintings, then combine these synthetic images with real paintings to train classifiers. Their hybrid approach—mixing real and synthetic data—outperforms models trained on real paintings alone, achieving higher ROC-AUC and accuracy scores. The key finding is that synthetic data generated from generative models can meaningfully augment limited real-world datasets for art authentication tasks.
Core Technical Contribution
The core contribution is demonstrating that synthetic images generated through DreamBooth fine-tuning can serve as effective data augmentation for painting authorship attribution. Unlike standard data augmentation (which applies geometric or color transforms to existing images), this approach generates entirely new synthetic paintings that capture artist-specific styles, effectively expanding the training distribution. The authors show that combining real and synthetic data in a hybrid training pipeline improves generalization compared to using only real paintings. This bridges generative and discriminative modeling—using a generative model (Stable Diffusion) to create training data, then feeding that into standard discriminative classifiers—which is a relatively novel data augmentation strategy for this domain.
How It Works
The pipeline operates in three stages: (1) Fine-tuning Stable Diffusion using DreamBooth on a small set of real paintings from a target artist, which learns to associate a unique identifier token with that artist's style. (2) Generating synthetic paintings by prompting the fine-tuned model with descriptions of paintings in that artist's style, producing new synthetic artworks that preserve stylistic features. (3) Combining the original real paintings with the generated synthetic images to create an expanded training dataset, which is then used to train standard image classifiers (likely CNNs) for the authorship attribution task. The synthetic data effectively increases the effective training set size, allowing the discriminative model to learn more robust feature representations of each artist's unique style without requiring additional real paintings.
Production Impact
In production art authentication systems, this approach directly solves the cold-start problem: when a museum or auction house has only a handful of paintings from an artist they need to authenticate, synthetic augmentation can boost model accuracy without waiting to collect more real examples. Teams building such systems would add a generative augmentation pipeline before their standard training loop—fine-tune Stable Diffusion on available real examples, generate synthetic variants, and merge them with real data before training the discriminator. Trade-offs include: computational cost for fine-tuning and generation (typically hours per artist on modern GPUs), potential domain shift issues if synthetic images diverge stylistically from real paintings, and the need to validate that synthetic augmentation doesn't introduce spurious correlations that fail on truly novel test paintings. Integration is relatively straightforward—this plugs into existing data preprocessing pipelines—but requires careful validation that synthetic data genuinely improves generalization rather than just overfitting to Stable Diffusion artifacts.
Limitations and When Not to Use This
The paper assumes that fine-tuned Stable Diffusion can capture the nuanced, high-frequency stylistic details that distinguish one painter from another, but this may fail for artists with subtle signature techniques or highly variable styles within their own body of work. The approach requires enough real training examples to successfully fine-tune DreamBooth (typically 3-5 high-quality images per artist), which may not exist for very obscure or historical painters. There is a risk of synthetic data leakage: if the generative model memorizes or partially reproduces specific famous paintings used in fine-tuning, classifiers may learn to recognize memorized visual patterns rather than generalizing to new authentic works. The paper likely does not address robustness to adversarial inputs, out-of-distribution paintings from unknown artists, or failure modes where synthetic data degrades performance on certain painting types (e.g., heavily damaged or heavily restored works).
Research Context
This work builds on the emerging intersection of synthetic data generation and computer vision tasks, building directly on DreamBooth (Ruiz et al., 2022) and Stable Diffusion. It extends prior data augmentation research in art analysis—which previously relied on simple transforms—into the generative era, showing that learned generative priors can encode style in ways useful for downstream discrimination tasks. The paper contributes to the broader artwork authentication and attribution literature, which traditionally relies on expert analysis or hand-crafted features. This opens a research direction toward using foundation models (Stable Diffusion, DALL-E, etc.) as style encoders for specialized domains where real data is scarce, with applications to other fine-grained classification tasks beyond painting attribution.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::