SkillOS: Learning Skill Curation for Self-Evolving Agents
:::info Stub — Full Engineering Breakdown Coming This paper was featured on Hugging Face Daily Papers on 2026-05-07 with 33 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Siru Ouyang et al. |
| Year | 2026 |
| HF Upvotes | 33 |
| arXiv | 2605.06614 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
LLM-based agents are increasingly deployed to handle streaming tasks, yet they often remain one-off problem solvers that fail to learn from past interactions. Reusable skills distilled from experience provide a natural substrate for self-evolution, where high-quality skill curation serves as the key bottleneck. Existing approaches either rely on manual skill curation, prescribe heuristic skill operations, or train for short-horizon skill operations. However, they still struggle to learn complex long-term curation policies from indirect and delayed feedback. To tackle this challenge, we propose SkillOS, an experience-driven RL training recipe for learning skill curation in self-evolving agents. SkillOS pairs a frozen agent executor that retrieves and applies skills with a trainable skill curator that updates an external SkillRepo from accumulated experience. To provide learning signals for curation, we design composite rewards and train on grouped task streams based on skill-relevant task dependencies, where earlier trajectories update the SkillRepo, and later related tasks evaluate these updates. Across multi-turn agentic tasks and single-turn reasoning tasks, SkillOS consistently outperforms memory-free and strong memory-based baselines in both effectiveness and efficiency, with the learned skill curator generalizing across different executor backbones and task domains. Further analyses show that the learned curator produces more targeted skill use, while the skills in SkillRepo evolve into more richly structured Markdown files that encode higher-level meta-skills over time.
Engineering Breakdown
Plain English
SkillOS is a training framework that teaches LLM-based agents to autonomously curate and evolve reusable skills from their own experience, rather than relying on manually created skill libraries. The system uses reinforcement learning to learn long-horizon skill curation policies—deciding when to create new skills, modify existing ones, or retire unused ones—from delayed, indirect feedback without requiring human annotation.
Key Engineering Insight
The core innovation is decoupling skill execution (frozen agent) from skill curation (trainable policy), allowing the system to learn complex multi-step curation decisions through RL without destabilizing the executor. This separation lets you train what skills to keep and how to organize them independent of how those skills get applied.
Why It Matters for Engineers
Production agents deployed on streaming tasks today accumulate skills ad-hoc and don't improve over time—you manually maintain skill libraries or rebuild agents. This work addresses the practical bottleneck: how to automatically decide which learned behaviors are worth keeping, when to consolidate them, and how to organize them. That directly cuts maintenance costs and improves agent performance without human intervention.
Research Context
Prior work either requires manual skill curation (not scalable) or uses fixed heuristics for skill management (brittle, inflexible). SkillOS advances the field by showing you can learn skill curation policies end-to-end with RL, handling long-horizon decisions and indirect rewards. This enables truly self-improving agents that adapt their skill libraries as task distributions shift—moving agents from static tool collections to dynamic, learned knowledge bases.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::