Vanast: Virtual Try-On with Human Image Animation via Synthetic Triplet Supervision
| Authors | Hyunsoo Cha et al. |
| Year | 2026 |
| HF Upvotes | 42 |
| arXiv | 2604.04934 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
We present Vanast, a unified framework that generates garment-transferred human animation videos directly from a single human image, garment images, and a pose guidance video. Conventional two-stage pipelines treat image-based virtual try-on and pose-driven animation as separate processes, which often results in identity drift, garment distortion, and front-back inconsistency. Our model addresses these issues by performing the entire process in a single unified step to achieve coherent synthesis. To enable this setting, we construct large-scale triplet supervision. Our data generation pipeline includes generating identity-preserving human images in alternative outfits that differ from garment catalog images, capturing full upper and lower garment triplets to overcome the single-garment-posed video pair limitation, and assembling diverse in-the-wild triplets without requiring garment catalog images. We further introduce a Dual Module architecture for video diffusion transformers to stabilize training, preserve pretrained generative quality, and improve garment accuracy, pose adherence, and identity preservation while supporting zero-shot garment interpolation. Together, these contributions allow Vanast to produce high-fidelity, identity-consistent animation across a wide range of garment types.
Engineering Breakdown
Plain English
Vanast is a unified framework that generates realistic human animation videos wearing different garments by taking a single human image, garment images, and a pose guidance video as inputs. Rather than treating virtual try-on and pose-driven animation as separate sequential steps (which causes identity drift, garment distortion, and inconsistency between front and back views), this approach performs the entire transformation in a single end-to-end step. The key innovation is constructing large-scale triplet supervision data that captures identity-preserving humans in alternative outfits along with full upper and lower garment pairs, overcoming the limitations of catalog images with single garments.
Core Technical Contribution
The core novelty is replacing the conventional two-stage pipeline (try-on then animation) with a single unified generation process that jointly optimizes for identity preservation, garment fidelity, and pose consistency. The authors introduced a triplet supervision strategy that generates synthetic training data containing (identity-preserved person in new outfit, original person image, garment catalog images), which provides richer learning signals than existing single-garment or two-stage approaches. This architectural unification directly addresses the compounding error problem where artifacts from stage one propagate into stage two, by constraining all objectives simultaneously during generation.
How It Works
The system takes three inputs: a source human image, one or more target garment images, and a pose guidance video that defines the motion sequence. The unified model processes these through a conditional generation pipeline that simultaneously performs garment transfer and pose-driven animation in a single forward pass, rather than first synthesizing the try-on result and then applying pose transformation separately. The training data pipeline generates large-scale triplets by creating identity-consistent variations of humans wearing different garments from a catalog, paired with the original image and extracted garment regions. During inference, the model leverages these learned correspondences to warp garments onto the target pose while maintaining consistent front-back relationships and preventing identity drift—likely through mechanisms like spatial feature alignment and attention-based garment warping that operate across the full pose sequence simultaneously.
Production Impact
For e-commerce and fashion platforms, this eliminates the need to maintain separate virtual try-on and animation models, reducing model serving costs and simplifying the inference pipeline. Production teams could offer dynamic fashion preview videos in a single inference call rather than chaining two models, reducing latency from ~4-6 seconds to potentially ~2-3 seconds depending on video length. The requirement for large-scale triplet supervision data is a significant hurdle—organizations would need to either purchase/generate ~100k+ triplets or fine-tune on proprietary fashion datasets, adding 2-4 weeks to pipeline setup. The unified approach makes end-to-end quality control harder since failure modes from multiple objectives are entangled; debugging whether a garment distortion comes from the try-on component or pose component becomes more complex than the modular two-stage approach.
Limitations and When Not to Use This
The paper doesn't clearly specify performance on edge cases like extreme poses, occlusions, or uncommon garment types not well-represented in the training triplets—likely these still fail gracefully to prior two-stage methods. Vanast requires full triplet supervision, meaning you cannot simply repurpose existing try-on or pose datasets independently; generating or acquiring this triplet data at scale is expensive and the paper provides limited guidance on synthetic data quality thresholds. The approach assumes relatively clean, centered human images and may struggle with crowded scenes, multiple people, or complex backgrounds where garment boundaries are ambiguous. Follow-up work still needs to address real-time inference (current approach likely requires GPU acceleration) and how to handle multiple concurrent pose sequences or batch video generation efficiently.
Research Context
This work builds on the established virtual try-on literature (which includes methods like VITON, CP-VTON) and pose-guided generation work, but challenges the fundamental assumption that these tasks should be separate stages. The contribution extends recent unified diffusion-based approaches for multi-task synthesis (similar to how text-to-image models unify multiple subtasks) into the fashion video domain. It opens a research direction toward end-to-end human content generation where identity, appearance, and motion are jointly modeled, which has implications beyond fashion for avatar animation, dance video synthesis, and digital human creation at scale.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::