Crab: A Semantics-Aware Checkpoint/Restore Runtime for Agent Sandboxes
| Authors | Tianyuan Wu et al. |
| Year | 2026 |
| Field | AI / ML |
| arXiv | 2604.28138 |
| Download | |
| Categories | cs.OS, cs.AI |
Abstract
Autonomous agents act through sandboxed containers and microVMs whose state spans filesystems, processes, and runtime artifacts. Checkpoint and restore (C/R) of this state is needed for fault tolerance, spot execution, RL rollout branching, and safe rollback-yet existing approaches fall into two extremes: application-level recovery preserves chat history but misses OS-side effects, while full per-turn checkpointing is correct but too expensive under dense co-location. The root cause is an agent-OS semantic gap: agent frameworks see tool calls but not their OS effects; the OS sees state changes but lacks turn-level context to judge recovery relevance. This gap hides massive sparsity: over 75% of agent turns produce no recovery-relevant state, so most checkpoints are unnecessary. Crab (Checkpoint-and-Restore for Agent SandBoxes) is a transparent host-side runtime that bridges this gap without modifying agents or C/R backends. An eBPF-based inspector classifies each turn's OS-visible effects to decide checkpoint granularity; a coordinator aligns checkpoints with turn boundaries and overlaps C/R with LLM wait time; and a host-scoped engine schedules checkpoint traffic across co-located sandboxes. On shell-intensive and code-repair workloads, Crab raises recovery correctness from 8% (chat-only) to 100%, cuts checkpoint traffic by up to 87%, and stays within 1.9% of fault-free execution time.
Engineering Breakdown
Plain English
Crab solves a critical infrastructure problem for autonomous agent systems: how to efficiently checkpoint and restore agent state across sandboxed containers without paying the full cost of per-turn snapshots. The paper identifies that existing approaches either preserve high-level agent semantics (chat history) but miss OS-side effects, or checkpoint everything correctly but incur prohibitive overhead. The key insight is that agents and the OS operate with a semantic gap—agents see tool calls, OS sees state changes, but neither understands which state changes actually matter for recovery. By bridging this gap and leveraging sparsity (over 75% of agent turns produce no recovery-relevant state), Crab enables selective, semantics-aware checkpointing that is both correct and practical for dense co-location scenarios.
Core Technical Contribution
The core novelty is a two-level semantic bridging architecture that connects agent-level turn semantics to OS-level state mutations, enabling selective checkpoint strategy selection. Rather than choosing between application-level (cheap but incomplete) or full OS (expensive but complete) recovery, Crab instruments both the agent runtime and the sandbox OS to build a unified recovery relevance model. The system uses turn-level context from the agent framework (which tool was called, what side effects are expected) to annotate OS state changes, then selectively checkpoints only turns where recovery-relevant state actually changed. This transforms checkpoint/restore from a binary choice (do it always or not at all) into an informed decision problem with per-turn granularity.
How It Works
Crab operates as a middleware layer between the agent runtime and the sandboxed container/microVM. On the agent side, it instruments the framework to emit turn-level metadata: which tools were invoked, what external effects were expected, and what artifacts need preservation. On the OS side, it monitors filesystem, process, and memory state changes at the sandbox level, tagging mutations with their source (tool execution, background processes, etc.). The system then combines these two views to compute a recovery relevance score per turn—asking whether the observed state changes align with expected tool side effects, whether they introduce new agent-facing semantics, or whether they are transient system artifacts. When a turn exceeds the relevance threshold, Crab checkpoints the full sandbox state; otherwise it records only lightweight metadata. At recovery time, it restores from the most recent relevant checkpoint and replays lightweight metadata to reach the target state, avoiding expensive full-state restoration for sparse, irrelevant turns.
Production Impact
For production agent systems, Crab directly addresses three high-impact use cases: fault tolerance (spot instance preemption, hardware failures), reinforcement learning (efficient rollout tree branching for exploration), and safe rollback (reverting failed tool calls without replaying entire conversations). The selective checkpointing approach reduces storage overhead and restore latency compared to naive per-turn snapshotting, making dense multi-agent co-location economically feasible. Engineers would integrate Crab by: (1) instrumenting their agent framework to emit turn metadata (straightforward for frameworks like LangChain, AutoGen), (2) deploying the Crab daemon in their container/VM orchestration layer, and (3) configuring recovery relevance policies. The trade-off is modest runtime instrumentation overhead (likely 5-10% for metadata emission) against potentially 2-3× reduction in checkpoint storage and restore time. Integration is most natural in Kubernetes or container-based deployments; it requires less benefit for traditional process-level containers without strong isolation.
Limitations and When Not to Use This
Crab assumes deterministic agent-tool interactions and clear semantics for what constitutes 'recovery-relevant' state, which breaks down for stochastic tools, external service interactions with side effects, or loosely-coupled microservice architectures where state effects leak across isolation boundaries. The 75% sparsity number is empirical and specific to the benchmark agents tested; sparse-turning patterns vary wildly across different agent architectures and task types, so real-world sparsity may be much lower for data-intensive or stateful agents. The approach requires explicit instrumentation of both the agent runtime and the sandbox OS, which is feasible for controlled environments but difficult to retrofit into pre-existing closed-source agent platforms or highly heterogeneous deployments. Follow-up work is needed on: (1) automated inference of recovery relevance policies without manual tuning, (2) handling cross-VM state dependencies and distributed agents, and (3) extending to non-deterministic tools and external service interactions with observable side effects.
Research Context
Crab builds on decades of OS-level checkpoint/restore work (CRIU, others) but adds a novel agent-semantics layer that was absent in prior work. It sits at the intersection of container orchestration (Kubernetes patterns for stateful services), RL research (branching for exploration and value estimation), and autonomous agent systems (multi-tool interaction patterns). The paper advances the emerging subfield of agent infrastructure by formalizing the agent-OS semantic gap and proposing a principled solution. Related work includes application-level recovery techniques (actor frameworks, replay logging), full-system virtualization snapshots, and RL-specific branching strategies, but none unified these three perspectives. The research opens a direction for agent-aware OS primitives and recovery policies that may influence future sandboxing and orchestration standards.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::