PhysMoDPO: Physically-Plausible Humanoid Motion with Preference Optimization
| Authors | Yangsong Zhang et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2603.13228 |
| Download | |
| Categories | cs.LG, cs.AI, cs.CV, cs.RO |
Abstract
Recent progress in text-conditioned human motion generation has been largely driven by diffusion models trained on large-scale human motion data. Building on this progress, recent methods attempt to transfer such models for character animation and real robot control by applying a Whole-Body Controller (WBC) that converts diffusion-generated motions into executable trajectories. While WBC trajectories become compliant with physics, they may expose substantial deviations from original motion. To address this issue, we here propose PhysMoDPO, a Direct Preference Optimization framework. Unlike prior work that relies on hand-crafted physics-aware heuristics such as foot-sliding penalties, we integrate WBC into our training pipeline and optimize diffusion model such that the output of WBC becomes compliant both with physics and original text instructions. To train PhysMoDPO we deploy physics-based and task-specific rewards and use them to assign preference to synthesized trajectories. Our extensive experiments on text-to-motion and spatial control tasks demonstrate consistent improvements of PhysMoDPO in both physical realism and task-related metrics on simulated robots. Moreover, we demonstrate that PhysMoDPO results in significant improvements when applied to zero-shot motion transfer in simulation and for real-world deployment on a G1 humanoid robot.
Engineering Breakdown
Plain English
This paper addresses a key problem in motion generation: diffusion models trained on text-to-motion data generate realistic human motions, but when a Whole-Body Controller (WBC) converts these motions into physically executable trajectories for robots or character animation, the output often deviates significantly from the original motion. PhysMoDPO solves this by using Direct Preference Optimization to train the diffusion model end-to-end with the WBC integrated into the training loop, eliminating the need for hand-crafted physics penalties like foot-sliding loss. Instead of treating physics compliance as a post-processing constraint, the method optimizes the diffusion model to generate motions that the WBC controller naturally preserves, keeping the motion close to the original intent while remaining physically plausible.
Core Technical Contribution
The core novelty is replacing hand-crafted physics-aware heuristics with a learned optimization framework that directly optimizes for WBC-compliant outputs. Rather than adding penalty terms to a loss function (the traditional approach), the authors integrated the full WBC physics controller into the diffusion training pipeline and applied Direct Preference Optimization (DPO)—a technique from RLHF that learns from comparative feedback without explicit reward modeling. This approach treats physics compliance as an emergent property of the learned diffusion model rather than an engineered constraint, allowing the model to discover its own solutions for balancing motion naturalness with physical feasibility. The key insight is that by optimizing the diffusion model to produce motions that a downstream physics controller preserves, you can eliminate the motion-physics gap without explicit hand-crafted objectives.
How It Works
The pipeline works in three stages. First, a base diffusion model is trained on large-scale text-to-motion data in standard fashion, learning to generate realistic human motions from text prompts. Second, for a given text prompt, the diffusion model generates a candidate motion trajectory. Third, this trajectory is fed into a Whole-Body Controller (WBC)—a physics-based controller that converts the idealized motion into physically feasible joint torques and contact constraints for bipedal locomotion. The WBC outputs a modified trajectory that satisfies physics constraints. Finally, the authors apply Direct Preference Optimization: they compare pairs of motions (original diffusion output vs. WBC-modified output) and train the diffusion model to generate motions in the first place that the WBC will preserve with minimal modification. This is formulated as a preference optimization problem where the 'preference' is how well the WBC output matches the original diffusion output, creating a feedback loop that teaches the diffusion model to output physics-aware motions natively.
Production Impact
For engineers building character animation or robot control systems, this dramatically simplifies the pipeline and reduces failure modes. Instead of tuning multiple loss terms (foot penetration penalty, contact force penalty, acceleration smoothness, etc.), you train once with WBC in the loop and get motions that work with your physics controller out of the box. This reduces manual hyperparameter tuning, eliminates the need for domain-specific physics knowledge when designing losses, and makes the system more robust to different character morphologies or robot dynamics—the WBC already handles those constraints. The trade-off is computational cost: every diffusion training step now requires running the WBC controller (a non-trivial simulation), which increases training time significantly compared to standard diffusion training. For deployment, inference time remains unchanged (just a diffusion forward pass), but the quality improvement means you can use fewer diffusion steps or lower-resolution models. Data requirements are similar to standard text-to-motion models, but you need accurate physics simulators and WBC implementations for your specific robot morphology.
Limitations and When Not to Use This
The method assumes you have a well-tuned Whole-Body Controller for your specific robot or character—if your WBC itself is poorly designed or brittle, the diffusion model will optimize for the wrong objective. The approach also doesn't handle sim-to-real transfer explicitly; physics fidelity in simulation may not translate to real-world robot execution due to model mismatch and friction uncertainties. The paper likely doesn't address edge cases where physics constraints are fundamentally incompatible with natural human motion (e.g., certain acrobatic moves), and in those cases the model must choose between naturalness and feasibility. Additionally, Direct Preference Optimization requires paired comparisons of motions, which adds data annotation overhead compared to standard supervised diffusion training. The method is also tied to the specific WBC formulation—if you change controllers, you may need to retrain, limiting generalizability across different robotic platforms.
Research Context
This work builds on recent advances in text-conditioned motion generation using diffusion models, which have become the dominant approach following works like MDM and MotionDiffuse. It extends the idea of integrating downstream task constraints into training—similar to how RLHF integrates human preferences into language model training. The paper directly addresses a gap identified in prior work: that converting diffusion outputs to physics-compliant trajectories via post-processing WBC introduces motion corruption. Related work includes motion style transfer, physics-guided animation synthesis, and robotic imitation learning, but PhysMoDPO is novel in applying Direct Preference Optimization specifically to the diffusion-WBC integration problem. This likely opens a research direction in 'constraint-aware diffusion training' where other downstream controllers or task-specific constraints could be integrated similarly during training rather than handled as post-hoc corrections.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::