MoRight: Motion Control Done Right
| Authors | Shaowei Liu et al. |
| Year | 2026 |
| HF Upvotes | 6 |
| arXiv | 2604.07348 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Generating motion-controlled videos--where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints--demands two capabilities: (1) disentangled motion control, allowing users to separately control the object motion and adjust camera viewpoint; and (2) motion causality, ensuring that user-driven actions trigger coherent reactions from other objects rather than merely displacing pixels. Existing methods fall short on both fronts: they entangle camera and object motion into a single tracking signal and treat motion as kinematic displacement without modeling causal relationships between object motion. We introduce MoRight, a unified framework that addresses both limitations through disentangled motion modeling. Object motion is specified in a canonical static-view and transferred to an arbitrary target camera viewpoint via temporal cross-view attention, enabling disentangled camera and object control. We further decompose motion into active (user-driven) and passive (consequence) components, training the model to learn motion causality from data. At inference, users can either supply active motion and MoRight predicts consequences (forward reasoning), or specify desired passive outcomes and MoRight recovers plausible driving actions (inverse reasoning), all while freely adjusting the camera viewpoint. Experiments on three benchmarks demonstrate state-of-the-art performance in generation quality, motion controllability, and interaction awareness.
Engineering Breakdown
Plain English
MoRight is a framework for generating videos where users can control object motion and camera viewpoint independently while maintaining physical plausibility. The paper identifies two critical limitations in existing methods: they conflate camera and object motion into a single control signal, and they treat motion as simple pixel displacement without modeling how actions cause reactions between objects. The authors propose a unified framework using disentangled motion modeling where object motion is specified in a canonical space separately from camera parameters, enabling coherent physical interactions between scene elements. This separation allows users to independently adjust what objects do and where the camera looks while preserving causal relationships between actions and their effects.
Core Technical Contribution
The core innovation is a disentangled motion representation that separates object dynamics from camera control in a canonical coordinate frame rather than entangling them in a single tracking signal. Instead of treating motion as kinematic displacement (pixel-level offsets), MoRight models motion causality—the physical cause-and-effect relationships where one object's action triggers coherent reactions from other objects in the scene. The framework uses a unified architecture that independently processes object motion specifications and camera viewpoints, then combines them during rendering to maintain physical plausibility. This is fundamentally different from prior tracking-based methods that optimize a single motion signal to move objects in image space without explicit causal modeling.
How It Works
The system takes three inputs: a source video, user-specified object motion (expressed in canonical 3D space), and desired camera parameters (viewpoint). The framework first disentangles these inputs by representing object motion in a canonical coordinate frame decoupled from camera trajectories—this prevents camera movement from accidentally changing object positions or vice versa. For each frame, the system computes how the specified object motion should manifest physically, including induced reactions from other scene elements (e.g., if one object pushes another, the second object moves accordingly). The motion causality component models these physical interactions rather than just displacing pixels, ensuring that actions propagate through the scene with proper cause-and-effect relationships. Finally, the renderer combines the disentangled object dynamics and camera parameters to generate the output video, maintaining consistency between user intent and physical realism across freely chosen viewpoints.
Production Impact
For teams building video generation or video editing systems, MoRight eliminates the need for separate pipelines handling camera control and object motion—a single framework handles both. This substantially reduces engineering complexity when building user-facing tools where creators want independent control over 'what moves' and 'where we're looking.' The disentangled motion modeling would improve usability in interactive applications: users wouldn't experience unintuitive camera-object coupling where adjusting viewpoint inadvertently repositions objects, or vice versa. However, adoption requires careful integration with existing rendering engines and may increase inference latency due to the additional causal relationship computation—teams should benchmark against their current approach. Data requirements are significant since the model must learn both motion representation and causal physics, potentially requiring larger training datasets than simpler tracking-based alternatives.
Limitations and When Not to Use This
The paper's abstract doesn't fully specify how the framework handles complex multi-body interactions or scenarios with many objects—cases where computing full causal chains becomes computationally expensive. The approach assumes a clear canonical coordinate frame exists for the scene, which may not hold for videos with complex deformable objects, fluid dynamics, or significant occlusions. There's no discussion of failure modes when user-specified motion violates physical constraints (e.g., requesting an object move through a wall), nor how the system handles ambiguous causality scenarios where multiple interpretations of cause-and-effect could apply. The method likely requires carefully curated training data where ground truth causal relationships are annotated, raising questions about scalability to diverse real-world videos where causality is subjective or emergent.
Research Context
This work builds on the broader literature of controllable video generation and physics-aware synthesis, extending prior art that focused on either camera control or object motion but not both simultaneously. It addresses limitations in tracking-based video editing methods (which conflate motion signals) and kinematic-only approaches (which ignore physics and causality). The research connects to work on disentangled representations in generative models and physics simulation, borrowing insights about separating controllable factors. This opens a research direction toward 'causal video generation' where models explicitly reason about action-reaction relationships rather than implicit pixel-level patterns, potentially leading to more interpretable and user-controllable video systems.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::