FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customization
:::info Stub — Full Engineering Breakdown Coming This paper has a linked code implementation and was featured on Hugging Face Papers with 15 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Quanjian Song et al. |
| Year | 2026 |
| HF Upvotes | 15 |
| arXiv | 2605.15824 |
| Download | |
| Code | https://github.com/quanjiansong/FashionChameleon |
Abstract
Human-centric video customization, particularly at the garment level, has shown significant commercial value. However, existing approaches cannot support low-latency and interactive garment control, which is crucial for applications such as e-commerce and content creation. This paper studies how to achieve interactive multi-garment video customization while preserving motion coherence using only single-garment video data. We present FashionChameleon, a real-time and interactive framework for human-garment customization in autoregressive video generation, where users can interactively switch garment during generation. FashionChameleon consists of three key techniques: (i) Instead of training on multi-garment video data, we train a Teacher Model with In-Context Learning on a single reference-garment pair. By retaining the image-to-video training paradigm while enforcing a mismatch between the reference and garment image, the model is encouraged to implicitly preserve coherence during single-garment switching. (ii) To achieve consistency and efficiency during generation, we introduce Streaming Distillation with In-Context Learning, which fine-tunes the model with in-context teacher forcing and improves extrapolation consistency via gradient-reweighted distribution matching distillation. (iii) To extend the model for interactive multi-garment video customization, we propose Training-Free KV Cache Rescheduling, which includes garment KV refresh, historical KV withdraw, and reference KV disentangle to achieve garment switching while preserving motion coherence. Our FashionChameleon uniquely supports interactive customization and consistent long-video extrapolation, while achieving real-time generation at 23.8 FPS on a single GPU, 30-180times faster than existing baselines.
Engineering Breakdown
Plain English
FashionChameleon enables real-time, interactive garment swapping in video generation—letting users change what a person is wearing while a video is being generated, without latency. The key trick: train on single-garment video data but use in-context learning in a teacher model to handle multi-garment customization, avoiding the need for expensive multi-garment training datasets.
Key Engineering Insight
Using in-context learning to transfer single-garment training to multi-garment control is the core innovation—it trades off computational cost at training time for architectural elegance, letting you generalize to unseen garment combinations without retraining. This is a practical data efficiency win for production systems.
Why It Matters for Engineers
E-commerce and content creation desperately need sub-100ms latency for interactive try-on experiences. Most video generation models can't handle garment-level control or require massive multi-garment datasets that don't exist at scale. This paper addresses both constraints: it works with readily available single-garment video and runs fast enough for real-time user interaction.
Research Context
Prior work on video customization either focused on non-interactive batch generation or required paired multi-garment datasets. FashionChameleon advances the state by making autoregressive video generation interactive (users can steer generation mid-way) and data-efficient (single-garment training generalizes). This unlocks a new class of interactive video editing tools that weren't feasible before.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::