daVinci-Env: Open SWE Environment Synthesis at Scale
| Authors | Dayuan Fu et al. |
| Year | 2026 |
| Field | AI / ML |
| arXiv | 2603.13023 |
| Download | |
| Categories | cs.SE, cs.AI, cs.CL |
Abstract
Training capable software engineering (SWE) agents demands large-scale, executable, and verifiable environments that provide dynamic feedback loops for iterative code editing, test execution, and solution refinement. However, existing open-source datasets remain limited in scale and repository diversity, while industrial solutions are opaque with unreleased infrastructure, creating a prohibitive barrier for most academic research groups. We present OpenSWE, the largest fully transparent framework for SWE agent training in Python, comprising 45,320 executable Docker environments spanning over 12.8k repositories, with all Dockerfiles, evaluation scripts, and infrastructure fully open-sourced for reproducibility. OpenSWE is built through a multi-agent synthesis pipeline deployed across a 64-node distributed cluster, automating repository exploration, Dockerfile construction, evaluation script generation, and iterative test analysis. Beyond scale, we propose a quality-centric filtering pipeline that characterizes the inherent difficulty of each environment, filtering out instances that are either unsolvable or insufficiently challenging and retaining only those that maximize learning efficiency. With 576K on trajectory sampling and difficulty-aware curation, the entire project represents a total investment of approximately $1.47 million, yielding about 13,000 curated trajectories from roughly 9,000 quality guaranteed environments. Extensive experiments validate OpenSWE's effectiveness: OpenSWE-32B and OpenSWE-72B achieve 62.4% and 66.0% on SWE-bench Verified, establishing SOTA among Qwen2.5 series. Moreover, SWE-focused training yields substantial out-of-domain improvements, including up to 12 points on mathematical reasoning and 5 points on science benchmarks, without degrading factual recall.
Engineering Breakdown
Plain English
This paper introduces OpenSWE, a large-scale framework for training software engineering AI agents with 45,320 executable Docker environments spanning over 12,800 repositories. The core problem is that existing SWE agent training datasets are small, limited in diversity, and proprietary—making it difficult for academic researchers to build and evaluate AI coding agents at scale. The authors solved this by building a fully open-sourced infrastructure with all Dockerfiles, evaluation scripts, and training data publicly available, deployed across a 64-node cluster. This enables reproducible, large-scale training of SWE agents on real Python repositories with genuine dynamic feedback loops for code iteration and testing.
Core Technical Contribution
The primary innovation is a multi-agent synthesis pipeline that automatically generates executable, containerized environments at scale—moving beyond static code snapshots to dynamic, testable sandboxes. Rather than manually curating datasets, the authors developed automation to extract repository structure, dependencies, and test harnesses directly into Docker containers, then validated that each environment executes correctly. This is fundamentally different from prior work like SWE-bench or HumanEval, which provide static code snippets; OpenSWE provides fully functional execution environments where agents can run real test suites and receive genuine feedback. The framework's transparency—with all infrastructure open-sourced—eliminates the reproducibility barrier that exists with proprietary industrial solutions like GitHub Copilot's training pipeline.
How It Works
The system takes as input a large corpus of Python repositories (12,800+ repos) and, for each, extracts metadata including dependencies, test configurations, and code structure. A multi-agent synthesis pipeline then automatically generates a Dockerfile for each repository, capturing the exact environment needed to run code and tests in isolation. These Dockerfiles are validated through actual test execution—if tests fail or the container doesn't build, the pipeline resynthesize or flags the environment as invalid. The resulting 45,320 validated Docker images become training environments where SWE agents receive live feedback: they can edit code, run test suites, see pass/fail results, and iterate. The entire infrastructure is deployed on a 64-node cluster, allowing parallel training and evaluation of multiple agent instances against different repository environments simultaneously.
Production Impact
For teams building SWE agents or code-generation models, this framework eliminates the months of work required to build reproducible evaluation infrastructure. Instead of engineering custom test harnesses and environment setup, engineers can directly train on validated, containerized repositories with guaranteed reproducibility—critical for comparing agent performance across model versions. The open-source nature means smaller research teams and startups can now afford to train capable SWE agents, reducing the competitive advantage of well-resourced labs. However, the trade-offs are significant: maintaining 45k+ Docker environments requires substantial compute (the 64-node cluster), storage overhead for 12.8k repositories, and ongoing effort to keep environments up-to-date as dependencies evolve. For production deployment, teams must decide whether to use OpenSWE environments directly (simplest but heavyweight) or extract the methodology to build lightweight, specialized evaluation suites for their specific codebase patterns.
Limitations and When Not to Use This
The framework is currently limited to Python repositories, excluding the majority of production SWE workloads in Java, C++, Go, and TypeScript—requiring separate infrastructure investments for other languages. The approach assumes repositories have standard test structures (pytest, unittest, etc.) and may fail silently or require manual fixes for repositories with non-standard CI/CD patterns or missing test coverage. Docker containerization adds latency compared to in-process execution, which could be problematic for agents requiring sub-second feedback loops. Additionally, the paper does not address how environment drift occurs over time as upstream dependencies release updates, security patches, or breaking changes—a critical issue for long-term reproducibility in production systems. The open-source release also raises questions about licensing and legal liability if agents trained on this data are used commercially.
Research Context
This work extends the SWE agent research trajectory established by papers like SWE-bench and related work on code-LLM evaluation, but at dramatically larger scale and with full reproducibility. Prior SWE agent benchmarks (SWE-bench, HumanEval) provided static snapshots of code problems; OpenSWE enables agents to interact with real dependency graphs and test suites, raising the bar for agent capability assessment. The multi-agent synthesis pipeline draws from infrastructure automation research and container orchestration techniques, applying them to the novel domain of synthetic environment generation at scale. This opens new research directions: long-horizon planning agents that must navigate complex, multi-file codebases; agents that learn to optimize for test-driven development patterns; and comparative studies of different agent architectures trained on identical, reproducible substrates.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::