Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies
:::info Stub — Full Engineering Breakdown Coming This paper was featured on Hugging Face Daily Papers on 2026-05-05 with 6 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Zirui Tang et al. |
| Year | 2026 |
| HF Upvotes | 6 |
| arXiv | 2605.03596 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Workspace learning requires AI agents to identify, reason over, exploit, and update explicit and implicit dependencies among heterogeneous files in a worker's workspace, enabling them to complete both routine and advanced tasks effectively. Despite its importance, existing relevant benchmarks largely evaluate agents on pre-specified or synthesized files with limited real-world dependencies, leaving workspace-level evaluation underexplored. To this end, we introduce Workspace-Bench, a benchmark for evaluating AI agents on Workspace Learning invOlving Large-Scale File Dependencies. We construct realistic workspaces with 5 worker profiles, 74 file types, 20,476 files (up to 20GB) and curate 388 tasks, each with its own file dependency graph, evaluated across 7,399 total rubrics that require cross-file retrieval, contextual reasoning, and adaptive decision-making. We further provide Workspace-Bench-Lite, a 100-task subset that preserves the benchmark distribution while reducing evaluation costs by about 70%. We evaluate 4 popular agent harnesses and 7 foundation models. Experimental results show that current agents remain far from reliable workspace learning, where the best reaches only 68.7%, substantially below the human result of 80.7%, and the average performance across agents is only 47.4%.
Engineering Breakdown
Plain English
This paper introduces Workspace-Bench, a benchmark for evaluating AI agents on realistic file management tasks in actual work environments. The benchmark includes 20,476 real files across 74 file types, 388 curated tasks with explicit dependency graphs, and 7,399 evaluation rubrics—significantly more complex and realistic than existing benchmarks that use pre-specified or synthetic files.
Key Engineering Insight
AI agents need to understand implicit and explicit dependencies between heterogeneous files to complete real workspace tasks, not just execute isolated commands. This means your agent evaluation needs to test dependency reasoning across file types, not just individual file operations.
Why It Matters for Engineers
Current AI agent benchmarks (like SWE-Bench or code-specific evals) don't capture the file dependency complexity that knowledge workers face daily—spreadsheets linked to databases, config files affecting multiple projects, documentation dependencies on code. If you're building agents for real office work, testing on synthetic or isolated tasks will miss critical failure modes in production.
Research Context
Prior benchmarks evaluated agents on pre-specified file sets or controlled synthetic environments. Workspace-Bench advances this by providing the first large-scale realistic benchmark with actual file dependency graphs at production scale (20GB, 5 worker personas). This enables proper evaluation of agents that need to reason about file relationships—a prerequisite for shipping agents into real organizational environments.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::