RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots
| Authors | Soroush Nasiriany et al. |
| Year | 2026 |
| Field | AI / ML |
| arXiv | 2603.04356 |
| Download | |
| Categories | cs.RO, cs.AI, cs.LG |
Abstract
Recent advances in robot learning have accelerated progress toward generalist robots that can perform everyday tasks in human environments. Yet it remains difficult to gauge how close we are to this vision. The field lacks a reproducible, large-scale benchmark for systematic evaluation. To fill this gap, we present RoboCasa365, a comprehensive simulation benchmark for household mobile manipulation. Built on the RoboCasa platform, RoboCasa365 introduces 365 everyday tasks across 2,500 diverse kitchen environments, with over 600 hours of human demonstration data and over 1600 hours of synthetically generated demonstration data -- making it one of the most diverse and large-scale resources for studying generalist policies. RoboCasa365 is designed to support systematic evaluations for different problem settings, including multi-task learning, robot foundation model training, and lifelong learning. We conduct extensive experiments on this benchmark with state-of-the-art methods and analyze the impacts of task diversity, dataset scale, and environment variation on generalization. Our results provide new insights into what factors most strongly affect the performance of generalist robots and inform strategies for future progress in the field.
Engineering Breakdown
Plain English
RoboCasa365 is a large-scale simulation benchmark for evaluating robot learning on household manipulation tasks. The authors built a platform with 365 everyday kitchen tasks across 2,500 diverse environments, backed by over 600 hours of human demonstrations and 1,600 hours of synthetic demonstrations. This addresses a critical gap in the field: there was no reproducible, large-scale benchmark to systematically measure progress toward generalist robots that can handle real-world household tasks. The resource aims to accelerate research on policies that generalize across diverse kitchen scenarios and task types.
Core Technical Contribution
The core contribution is RoboCasa365 itself—a comprehensive, open-source simulation benchmark that combines scale, diversity, and high-quality demonstration data in one resource. Unlike prior benchmarks that focus on single tasks or limited environments, RoboCasa365 spans 365 tasks across 2,500 kitchen variations, making it one of the largest and most diverse robot learning datasets available. The key novelty is the systematic combination of human-annotated demonstrations (600+ hours) with synthetically generated data (1,600+ hours), enabling researchers to study how policies scale with demonstration diversity and volume. This enables reproducible evaluation of generalist robot policies on a standardized platform rather than ad-hoc single-task evaluations.
How It Works
RoboCasa365 is built on the RoboCasa simulation platform, which provides a physics-based kitchen environment with manipulable objects, mobile manipulation robots, and task specification interfaces. The benchmark defines 365 distinct household tasks (e.g., placing dishes, opening cabinets, fetching items) that are distributed across 2,500 procedurally varied kitchen layouts with different object positions, cabinet configurations, and spatial arrangements. For each task, the platform includes human demonstration trajectories (600+ hours collected) that show optimal or near-optimal performance, plus synthetically generated demonstrations (1,600+ hours) created through simulation rollouts or policy-based generation to increase data diversity. Researchers can train policies on varying subsets of tasks and demonstrations, then evaluate generalization across held-out tasks, environments, and demonstration counts, enabling systematic ablations on factors affecting generalist performance.
Production Impact
For teams building real robot systems, RoboCasa365 provides a standardized evaluation harness to measure how well policies trained on simulation transfer and generalize before expensive real-world testing. This reduces the chicken-and-egg problem: you can now benchmark your learning algorithm against a large, diverse corpus rather than building custom evaluation suites for each task domain. In a production pipeline, you would use RoboCasa365 to validate that your policy generalizes across task variations and unseen kitchen layouts before committing to real-world data collection and hardware deployment. The trade-off is that simulation-to-reality transfer still requires careful domain randomization and sim2real techniques—RoboCasa365 doesn't eliminate the reality gap, but it lets you measure generalization in a controlled setting. The large demonstration dataset (2,200 hours total) also enables researchers to study learning sample efficiency and the value of synthetic vs. human data, informing data collection strategies in production systems.
Limitations and When Not to Use This
RoboCasa365 is a simulation benchmark, so it inherits the core limitation that policies trained purely in simulation may fail to transfer to physical robots due to visual, tactile, and dynamics mismatches that simulation cannot fully capture. The benchmark focuses on kitchen manipulation, so results on RoboCasa365 do not directly guarantee performance on other household domains (laundry, cleaning, organizing living rooms) or outdoors, limiting the scope of 'generalist' claims. The paper does not provide full results showing how well policies trained on RoboCasa365 transfer to real robots—this is crucial validation that is deferred to future work. Additionally, the benchmark assumes access to high-quality human demonstrations (600+ hours), which is expensive to collect; the paper's analysis of how performance degrades with fewer demonstrations is not detailed in the abstract, so the practical data requirements for useful performance remain unclear.
Research Context
RoboCasa365 builds on prior work in robot learning benchmarks (like CALVIN, Franka Kitchen, and Metaworld) but scales up significantly in task diversity and demonstration quantity, moving the field toward more realistic household scenarios. It contributes to the broader push for 'generalist' or 'foundation' robot policies—inspired by the success of large language models and vision models—by providing a large, diverse dataset needed to train and evaluate such models. The benchmark enables the next generation of robot learning research to move beyond single-task or single-domain evaluation toward systematic studies of scaling laws, generalization, and learning efficiency in embodied AI. This work sits at the intersection of imitation learning (learning from demonstrations), simulation environments for robotics, and benchmark design, and it will likely become a standard evaluation platform for the household manipulation community.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::