The Last Human-Written Paper: Agent-Native Research Artifacts
| Authors | Jiachen Liu et al. |
| Year | 2026 |
| HF Upvotes | 13 |
| arXiv | 2604.24658 |
| Download | |
| Code | https://github.com/Orchestra-Research/Agent-Native-Research-Artifact |
Abstract
Scientific publication compresses a branching, iterative research process into a linear narrative, discarding the majority of what was discovered along the way. This compilation imposes two structural costs: a Storytelling Tax, where failed experiments, rejected hypotheses, and the branching exploration process are discarded to fit a linear narrative; and an Engineering Tax, where the gap between reviewer-sufficient prose and agent-sufficient specification leaves critical implementation details unwritten. Tolerable for human readers, these costs become critical when AI agents must understand, reproduce, and extend published work. We introduce the Agent-Native Research Artifact (ARA), a protocol that replaces the narrative paper with a machine-executable research package structured around four layers: scientific logic, executable code with full specifications, an exploration graph that preserves the failures compilation discards, and evidence grounding every claim in raw outputs. Three mechanisms support the ecosystem: a Live Research Manager that captures decisions and dead ends during ordinary development; an ARA Compiler that translates legacy PDFs and repos into ARAs; and an ARA-native review system that automates objective checks so human reviewers can focus on significance, novelty, and taste. On PaperBench and RE-Bench, ARA raises question-answering accuracy from 72.4% to 93.7% and reproduction success from 57.4% to 64.4%. On RE-Bench's five open-ended extension tasks, preserved failure traces in ARA accelerate progress, but can also constrain a capable agent from stepping outside the prior-run box depending on the agent's capabilities.
Engineering Breakdown
Plain English
This paper identifies a fundamental problem in how scientific research is published: traditional papers compress iterative, branching research into a linear narrative, discarding failed experiments and implementation details. The authors introduce Agent-Native Research Artifacts (ARA), a machine-executable package format designed to replace conventional papers by preserving the full research process—both the successful path and the exploration branches—in a format that AI agents can understand, reproduce, and extend. The core insight is that the 'Storytelling Tax' (discarded failed experiments) and 'Engineering Tax' (missing implementation details) are acceptable for human readers but become critical failure points when AI agents need to work with published research. ARA replaces narrative prose with structured, executable specifications that enable agents to reproduce results end-to-end and build on prior work without the information loss inherent in traditional papers.
Core Technical Contribution
The paper's primary contribution is reframing scientific publication as a machine-readability problem and proposing ARA as a structural solution. Instead of treating the paper as the primary artifact, ARA treats the executable research package as primary—including code, data specifications, experimental logs, failed hypotheses, and ablation studies—with human-readable summary as secondary documentation. This inverts the traditional publishing hierarchy where narrative prose is primary and implementation details are appendices or supplementary materials. The technical novelty lies in defining a protocol that captures research artifacts in agent-executable form while maintaining human interpretability, addressing the gap between what makes sense for human peer review and what makes sense for machine reproduction and extension.
How It Works
An ARA package contains several interlinked components: (1) a formal specification of the research hypothesis and success criteria in machine-readable form, (2) the complete experimental code with explicit dependency and version specifications, (3) the branching tree of experiments actually conducted (both successful and failed paths, not just the final narrative), (4) data provenance and generation specifications including train/test splits and random seeds, and (5) ablation studies and sensitivity analyses structured as executable queries rather than prose descriptions. When an AI agent receives an ARA, it can directly parse the hypothesis specification to understand what was being tested, execute the code to reproduce results with explicit handling of failed branches, and extend the work by modifying the experimental specification and re-running the package. The protocol includes a manifest file that maps each claim in any accompanying human-readable summary back to the executable evidence that supports it—creating auditability that goes beyond traditional citation. Crucially, the failed experiments and exploration branches are not discarded but stored as alternate execution paths, allowing agents to learn from the entire research process rather than just the published outcome.
Production Impact
For teams building AI systems that consume research, ARA enables a qualitative shift from 'reading papers and reimplementing from scratch' to 'directly executing and extending research artifacts.' This reduces the barrier to reproduction from weeks of engineering work to running a containerized package, dramatically accelerating research velocity in production pipelines. Engineering teams can programmatically query the branching experiment tree to understand not just 'does this method work' but 'under what conditions does it fail' and 'what hyperparameter choices were tested'—information that's typically scattered across appendices or not published at all. Integration with AI agents means that systems can autonomously consume new research by executing ARAs, running comparative benchmarks against production baselines, and suggesting whether adoption is warranted—eliminating the manual research review bottleneck. The trade-off is that authors must structure their work more rigorously upfront (defining specifications, tracking failed experiments systematically, containerizing code), which increases publication overhead, but this cost is borne once per research project and amortized across all future consumption by agents.
Limitations and When Not to Use This
The paper assumes that research can be meaningfully represented as executable specifications with clear success criteria, which breaks down for exploratory research where hypotheses are not pre-specified or success is defined qualitatively. ARA requires that code and data be reproducible and containerizable—a significant barrier for research using proprietary datasets, closed-source tools, or hardware-dependent experiments (e.g., robotics, large-scale distributed training where exact reproduction is expensive or impossible). The protocol likely struggles with research that involves human judgment calls (e.g., qualitative user studies, design choices made iteratively) that don't naturally decompose into machine-executable steps. Additionally, the paper doesn't address adversarial considerations—a malicious actor could encode training data extraction or model theft into an ARA package that appears to be a research artifact, requiring significant sandboxing and verification infrastructure that isn't discussed. The current work is presented as a protocol proposal, but without evidence of adoption by major conferences or publishers, it's unclear whether the ecosystem shifts necessary to make ARA standard are feasible or whether the overhead will limit uptake.
Research Context
This work sits at the intersection of reproducibility research, machine-readable science, and the emerging field of AI agents consuming research. It builds on decades of concern about irreproducibility in ML (where many papers cannot be reproduced even with author assistance) and extends critiques of narrative-driven publishing by authors like Akhlaq et al. and the Distill team, who have advocated for richer interactive media in scientific communication. The paper is implicitly responding to the scaling of AI agents—as systems like automated research tools, meta-learners, and AI scientists become more capable, the information loss inherent in text-only papers becomes a bottleneck. ARA can be understood as designing for machine readership the way literate programming (Knuth) designed code for human understanding, or as applying the 'source-of-truth' principle from software engineering (code is the source of truth, not documentation) to scientific publication. The work opens a research direction around standardizing machine-readable science artifacts and designing peer review processes that validate both narrative claims and executable specifications.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::