SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agents
:::info Stub — Full Engineering Breakdown Coming This paper was featured on Hugging Face Daily Papers on 2026-04-19 with 22 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Ziao Zhang et al. |
| Year | 2026 |
| HF Upvotes | 22 |
| arXiv | 2604.17308 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
As the capability frontier of autonomous agents continues to expand, they are increasingly able to complete specialized tasks through plug-and-play external skills. Yet current benchmarks mostly test whether models can use provided skills, leaving open whether they can discover skills from experience, repair them after failure, and maintain a coherent library over time. We introduce SkillFlow, a benchmark of 166 tasks across 20 families in which task construction within each family follows a Domain-Agnostic Execution Flow (DAEF) that defines an agent workflow framework, allowing these tasks to share a consistent workflow. Agents are evaluated under an Agentic Lifelong Learning protocol in which they begin without skills, solve tasks sequentially within each family, externalize lessons through trajectory- and rubric-driven skill patches, and carry the updated library forward. Experiments reveal a substantial capability gap. For Claude Opus 4.6, lifelong skill evolution improves task success from 62.65% to 71.08% (+8.43 points). However, high skill usage does not necessarily imply high utility: Kimi K2.5 gains only +0.60 points despite 66.87% skill usage, while Qwen-Coder-Next reaches only a 44.58% task completion rate and still regresses relative to the vanilla setting. SkillFlow contributes a structured testbed for this direction and an in-depth empirical analysis of skill discovery, patching, transfer, and their failure modes under lifelong evaluation.
Engineering Breakdown
Plain English
SkillFlow is a benchmark with 166 tasks across 20 families designed to test whether autonomous agents can discover, repair, and maintain skills over time—not just use pre-provided ones. The benchmark uses a Domain-Agnostic Execution Flow (DAEF) framework to create consistent task workflows, and evaluates agents under lifelong learning conditions where they start with no skills and must externalize lessons learned from sequential task solving.
Key Engineering Insight
The critical technical contribution is decoupling skill discovery and maintenance from skill execution: rather than testing if agents can invoke external tools, this benchmark forces agents to identify what skills they need, build them from experience, debug them after failures, and keep a coherent skill library over time—this is fundamentally harder than the current eval paradigm.
Why It Matters for Engineers
Production agents today fail silently when they lack the right tool or when tools break in new contexts; SkillFlow addresses this by measuring whether agents can diagnose skill gaps, repair broken abstractions, and evolve their tool libraries without human intervention—a critical capability for systems that can't rely on constant redeployment.
Research Context
Previous benchmarks measure skill usage (given tools, use them correctly), but real autonomous systems need skill discovery and evolution. SkillFlow advances the field by operationalizing lifelong learning as a measurable property, moving agents from static tool-use systems toward systems that can autonomously build and maintain their own capability libraries—a prerequisite for true long-horizon autonomous agents.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::