Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning
:::info Stub — Full Engineering Breakdown Coming This paper was featured on Hugging Face Daily Papers on 2026-05-07 with 65 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Yaorui Shi et al. |
| Year | 2026 |
| HF Upvotes | 65 |
| arXiv | 2605.06130 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in isolation or with separate reward sources, resulting in partial and conflicting evolution. We propose Skill1, a framework that trains a single policy to co-evolve skill selection, utilization, and distillation toward a shared task-outcome objective. The policy generates a query to search the skill library, re-ranks candidates to select one, solves the task conditioned on it, and distills a new skill from the trajectory. All learning derives from a single task-outcome signal. Its low-frequency trend credits selection and its high-frequency variation credits distillation. Experiments on ALFWorld and WebShop show that Skill1 outperforms prior skill-based and reinforcement learning baselines. Training dynamics confirm the co-evolution of the three capabilities, and ablations show that removing any credit signal degrades the evolution.
Engineering Breakdown
Plain English
Skill1 trains a single language model agent policy to simultaneously handle three interconnected tasks: selecting relevant skills from a library, using those skills to solve problems, and creating new skills from successful experiences. Instead of optimizing these three capabilities separately (which causes conflicts), the framework uses one unified reward signal tied to task outcomes, allowing the agent to learn how skills relate to each other naturally.
Key Engineering Insight
The core insight is that skill selection, skill execution, and skill distillation are interdependent—training them separately with different objectives creates conflicting gradients. By routing all three through a single task-outcome reward signal, the policy learns implicit dependencies that make skills more reusable and better adapted to actual task requirements.
Why It Matters for Engineers
Production agents that maintain skill libraries today face a scaling problem: they accumulate skills that work in isolation but don't compose well, or they waste compute re-learning variants of the same skill. Skill1 directly addresses reusability at the architecture level, which means fewer redundant skills in your library and faster adaptation to new tasks—both critical for keeping inference costs down in deployed multi-task systems.
Research Context
Prior work treated skill libraries as static components or optimized skill discovery/usage separately from skill creation, leading to libraries that grew inefficient over time. Skill1 advances the field by treating the skill library as a living, co-evolving system where the agent learns what skills matter, how to apply them, and when to create new ones—enabling the kind of continuous improvement that makes long-horizon agent deployment practical.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::