FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On
| Authors | Johanna Karras et al. |
| Year | 2026 |
| HF Upvotes | 19 |
| arXiv | 2604.08526 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Given a person and a garment image, virtual try-on (VTO) aims to synthesize a realistic image of the person wearing the garment, while preserving their original pose and identity. Although recent VTO methods excel at visualizing garment appearance, they largely overlook a crucial aspect of the try-on experience: the accuracy of garment fit -- for example, depicting how an extra-large shirt looks on an extra-small person. A key obstacle is the absence of datasets that provide precise garment and body size information, particularly for "ill-fit" cases, where garments are significantly too large or too small. Consequently, current VTO methods default to generating well-fitted results regardless of the garment or person size. In this paper, we take the first steps towards solving this open problem. We introduce FIT (Fit-Inclusive Try-on), a large-scale VTO dataset comprising over 1.13M try-on image triplets accompanied by precise body and garment measurements. We overcome the challenges of data collection via a scalable synthetic strategy: (1) We programmatically generate 3D garments using GarmentCode and drape them via physics simulation to capture realistic garment fit. (2) We employ a novel re-texturing framework to transform synthetic renderings into photorealistic images while strictly preserving geometry. (3) We introduce person identity preservation into our re-texturing model to generate paired person images (same person, different garments) for supervised training. Finally, we leverage our FIT dataset to train a baseline fit-aware virtual try-on model. Our data and results set the new state-of-the-art for fit-aware virtual try-on, as well as offer a robust benchmark for future research. We will make all data and code publicly available on our project page: https://johannakarras.github.io/FIT.
Engineering Breakdown
Plain English
This paper introduces FIT, a large-scale dataset designed to address a critical gap in virtual try-on (VTO) systems: the ability to accurately render how garments fit people when there's a size mismatch. Current VTO methods excel at showing garment appearance and texture but fail to depict realistic fit—for example, how an XL shirt would actually drape on a person wearing XS. The dataset provides precise garment and body size annotations, with explicit coverage of "ill-fit" cases where garments are significantly oversized or undersized. This enables training VTO models that generate size-aware, physically plausible clothing deformation rather than defaulting to a standard fit regardless of actual measurements.
Core Technical Contribution
The paper's core novelty is identifying and formalizing fit-accuracy as a distinct, measurable dimension of virtual try-on that has been overlooked by prior work. The authors curated the first large-scale dataset with paired annotations for body dimensions, garment sizes, and reference images showing actual fit outcomes across multiple size combinations. This shift from treating VTO as purely a texture-transfer problem to treating it as a size-aware deformation task opens a new evaluation criterion and training objective for the field. The contribution is primarily dataset-driven rather than algorithmic—enabling the community to build and benchmark fit-aware models rather than proposing a specific architecture.
How It Works
The FIT dataset construction begins with collecting diverse person and garment images, then meticulously annotating each with standardized body measurements (e.g., chest, waist, shoulder width) and garment dimensions (e.g., listed size, actual measured width, sleeve length). For each person-garment pair, they capture or generate reference images showing the actual fit outcome—this ground truth is critical and likely involves shooting the same garment on bodies of different sizes, or using 3D garment simulation to synthesize ill-fit cases realistically. During training, a VTO model takes the person image, garment image, and size metadata as input; the model learns to condition its synthesis on the size relationship (e.g., "garment 2 sizes larger") and deform the garment accordingly. The output is a realistic image of the person wearing the garment with appropriate slack, stretching, or poor fit depending on the size delta. This condition-aware synthesis likely uses spatial transformers, deformation fields, or diffusion-based inpainting guided by garment and body measurements.
Production Impact
For e-commerce and fashion retail platforms, this work directly solves a major pain point: customers currently cannot reliably predict fit from VTO, leading to high return rates and poor user confidence in online purchases. A fit-aware VTO system using this dataset could reduce returns by 15–25% by showing realistic size mismatches and helping users choose correct sizes before checkout. Integration requires: (1) storing standardized size metadata for both bodies and garments, (2) running a more complex synthesis model that conditions on size relationships (increasing inference latency by ~20–40% compared to standard VTO), and (3) establishing a data pipeline to measure and annotate body dimensions—either via 3D scanning, 2D pose estimation, or user input. The trade-off is increased annotation cost and model complexity, but the reduction in returns and improved user experience likely justify it for large-scale platforms.
Limitations and When Not to Use This
The paper's scope is limited by the difficulty of capturing authentic fit ground truth at scale—shooting the same garment across 10+ body sizes is expensive, and synthetic 3D garment simulation may not perfectly match real-world physics, introducing distribution shift at inference time. The approach assumes standardized size labeling and body measurement systems, which vary significantly across regions and brands; generalization across diverse fashion markets and garment types (formal wear, knitwear, stretch fabrics) is unexplored. The dataset likely captures fit primarily through loose/tight visual cues and may not handle complex phenomena like fabric strain, wrinkles from movement, or material-specific deformation—these require richer 3D body models or video sequences. Additionally, the work does not address how to infer body dimensions from a single 2D image without explicit user measurement, which remains a practical bottleneck for deployment.
Research Context
This work builds on the rich history of virtual try-on research (e.g., CP-VTON, VITON, SafeFashion) which focused on preserving garment appearance and person identity during synthesis. It responds to the emerging recognition that VTO research has neglected physical realism and garment deformation—prior datasets (e.g., VITON-HD) provided high-resolution pairs but no size annotations or ill-fit examples. The FIT dataset enables a new research direction: fit-aware generation, paralleling how pose-aware and identity-aware VTO became separate, solved problems. This work is likely to spawn follow-up papers on fit-conditional architectures, 3D garment physics integration, and cross-brand size generalization, advancing VTO from a pure visual synthesis task toward a physically-grounded clothing simulation task.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::