SafeGen-LLM: Enhancing Safety Generalization in Task Planning for Robotic Systems
| Authors | Jialiang Fan et al. |
| Year | 2026 |
| Field | AI / ML |
| arXiv | 2602.24235 |
| Download | |
| Categories | cs.RO, cs.AI |
Abstract
Safety-critical task planning in robotic systems remains challenging: classical planners suffer from poor scalability, Reinforcement Learning (RL)-based methods generalize poorly, and base Large Language Models (LLMs) cannot guarantee safety. To address this gap, we propose safety-generalizable large language models, named SafeGen-LLM. SafeGen-LLM can not only enhance the safety satisfaction of task plans but also generalize well to novel safety properties in various domains. We first construct a multi-domain Planning Domain Definition Language 3 (PDDL3) benchmark with explicit safety constraints. Then, we introduce a two-stage post-training framework: Supervised Fine-Tuning (SFT) on a constraint-compliant planning dataset to learn planning syntax and semantics, and Group Relative Policy Optimization (GRPO) guided by fine-grained reward machines derived from formal verification to enforce safety alignment and by curriculum learning to better handle complex tasks. Extensive experiments show that SafeGen-LLM achieves strong safety generalization and outperforms frontier proprietary baselines across multi-domain planning tasks and multiple input formats (e.g., PDDLs and natural language).
Engineering Breakdown
Plain English
This paper tackles a critical gap in robotic task planning: while classical planners don't scale, RL methods generalize poorly, and vanilla LLMs can't guarantee safety. SafeGen-LLM is a specialized language model that learns to generate provably safe task plans while generalizing to novel safety constraints across different domains. The authors built a multi-domain PDDL3 benchmark with explicit safety constraints and use a two-stage training pipeline combining supervised fine-tuning on constraint-compliant plans with a group relative policy optimization method to ensure both safety satisfaction and generalization.
Core Technical Contribution
The core novelty is a safety-aware fine-tuning framework specifically designed to teach LLMs to respect formal safety constraints during task planning, rather than treating safety as a post-hoc verification step. Unlike standard LLM fine-tuning which focuses on task performance, SafeGen-LLM explicitly optimizes for constraint satisfaction while maintaining generalization to unseen safety properties—achieved through a two-stage approach: SFT for planning syntax/semantics, then Group Relative Policy Optimization (GRPO) to align outputs with safety constraints. The key insight is treating safety constraint generalization as a learned capability rather than an enforcement mechanism, allowing the model to internalize what makes a plan safe across different domain configurations.
How It Works
SafeGen-LLM operates in two distinct training phases. First, Supervised Fine-Tuning (SFT) trains the base LLM on a curated dataset of valid PDDL3 task plans that comply with safety constraints, teaching the model to understand planning syntax, action sequences, and safety-aware reasoning. Second, Group Relative Policy Optimization (GRPO) uses constraint-satisfaction signals to further align the model, comparing generated plans against safety properties and optimizing the policy to maximize constraint compliance relative to baseline outputs. During inference, the model takes as input a task specification and target safety constraints in natural language or structured format, then generates a PDDL3 plan that should satisfy those constraints. The model essentially learns to internally represent and reason about safety trade-offs during generation, allowing it to handle novel constraint combinations without retraining.
Production Impact
In production robotic systems, SafeGen-LLM eliminates the need for expensive post-hoc constraint checking or replanning when safety violations occur—plans are generated safe-by-default, reducing latency in time-critical applications like autonomous manipulation or mobile robotics. Engineers can encode domain-specific safety rules (e.g., collision avoidance, energy limits, hardware constraints) into the training objective, and the model generalizes these principles to new scenarios without manual rule engineering. However, this approach requires significant upfront investment: building multi-domain PDDL3 datasets with ground-truth safe plans, computing the cost of two-stage fine-tuning on a specialized LLM (likely billions of parameters), and validating that the learned safety semantics hold across production environments where the training distribution assumptions may not hold. Integration challenges include converting domain-specific constraints into the format the model was trained on, handling distribution shift when deploying to robots with different morphologies or dynamics, and establishing formal verification that critical safety properties are never violated.
Limitations and When Not to Use This
The paper assumes availability of high-quality, constraint-compliant planning datasets across multiple domains—in practice, generating safe ground-truth PDDL3 plans for diverse robotics tasks is labor-intensive and may not cover all real-world edge cases. SafeGen-LLM likely inherits LLM hallucination risks where the model generates syntactically valid but semantically unsafe plans that don't actually satisfy the constraints in the true world; GRPO helps during training but doesn't guarantee zero failure rate in deployment. The approach is bounded by the expressivity of PDDL3: complex continuous dynamics, probabilistic effects, or multi-agent coordination may exceed what the model learns to represent. Additionally, the paper doesn't address how to handle constraint conflicts (e.g., two incompatible safety requirements) or formally verify that learned constraints hold under distribution shift—generalization to 'novel safety properties' is tested in their benchmark but may fail when real-world physics or robot capabilities differ significantly from training conditions.
Research Context
This work bridges three traditionally separate areas: classical symbolic planning (PDDL), reinforcement learning alignment (safety-conditioned policy learning), and large language model fine-tuning. It builds on recent advances in using LLMs for planning (like Gato and other foundation models applied to robotics) but specifically addresses the safety generalization problem that standard RL and LLM approaches struggle with. The multi-domain PDDL3 benchmark contribution will be valuable for the community to measure safety generalization, analogous to how benchmarks like MetaWorld or RoboNet have enabled progress in other robotics areas. Future research directions include formal verification of learned safety properties, combining SafeGen-LLM with world models for better grounding, and extending to continuous control domains beyond discrete task planning.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::