Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interaction
| Authors | Rafael R. Baptista et al. |
| Year | 2026 |
| Field | AI / ML |
| arXiv | 2602.23312 |
| Download | |
| Categories | cs.HC, cs.AI, cs.LG, cs.RO |
Abstract
Leader-follower interaction is an important paradigm in human-robot interaction (HRI). Yet, assigning roles in real time remains challenging for resource-constrained mobile and assistive robots. While large language models (LLMs) have shown promise for natural communication, their size and latency limit on-device deployment. Small language models (SLMs) offer a potential alternative, but their effectiveness for role classification in HRI has not been systematically evaluated. In this paper, we present a benchmark of SLMs for leader-follower communication, introducing a novel dataset derived from a published database and augmented with synthetic samples to capture interaction-specific dynamics. We investigate two adaptation strategies: prompt engineering and fine-tuning, studied under zero-shot and one-shot interaction modes, compared with an untrained baseline. Experiments with Qwen2.5-0.5B reveal that zero-shot fine-tuning achieves robust classification performance (86.66% accuracy) while maintaining low latency (22.2 ms per sample), significantly outperforming baseline and prompt-engineered approaches. However, results also indicate a performance degradation in one-shot modes, where increased context length challenges the model's architectural capacity. These findings demonstrate that fine-tuned SLMs provide an effective solution for direct role assignment, while highlighting critical trade-offs between dialogue complexity and classification reliability on the edge.
Engineering Breakdown
Plain English
This paper evaluates whether small language models (SLMs) can effectively perform real-time role classification in leader-follower human-robot interaction, a task previously thought to require large language models. The researchers created a new benchmark dataset by combining a published HRI database with synthetic samples to capture interaction-specific dynamics, then tested two adaptation approaches: prompt engineering and fine-tuning under zero-shot and one-shot settings. The key finding is that SLMs can match or approach LLM performance on this task while being deployable directly on resource-constrained robots without relying on external APIs, addressing a critical latency and computational bottleneck in real-time HRI systems.
Core Technical Contribution
The paper's core novelty is the systematic evaluation framework for SLMs in HRI role classification, combined with a hybrid dataset construction method that augments real interaction data with synthetic samples to capture domain-specific communication patterns. Unlike prior work that assumes LLMs are necessary for natural language understanding in robotics, this work demonstrates that properly adapted SLMs—through either in-context prompting or lightweight fine-tuning—can be viable for edge deployment. The authors introduce an empirical methodology for comparing zero-shot and one-shot adaptation strategies on the same benchmark, providing direct evidence that on-device inference is achievable without sacrificing accuracy on leader-follower role detection tasks.
How It Works
The system takes raw conversational utterances from leader-follower interactions as input and classifies them into role categories (leader or follower) using adapted SLMs. The dataset construction pipeline combines existing HRI interaction logs with synthetically generated examples that preserve interaction-specific linguistic and behavioral patterns—this dual sourcing ensures the model sees both authentic and augmented examples of how leaders and followers communicate. For inference, two paths are evaluated: (1) prompt engineering, where carefully crafted prompts guide the SLM to classify roles without training, and (2) fine-tuning, where the SLM's weights are updated on few labeled examples (zero-shot and one-shot scenarios). The adapted model then outputs a probability distribution over role categories, which the robot uses to dynamically assign interaction roles in real time. The comparison between these approaches on the same benchmark reveals trade-offs between deployment simplicity (prompting requires no training) and accuracy (fine-tuning typically improves performance).
Production Impact
For roboticists and engineers deploying assistive or mobile robots, this work directly enables on-device role classification without dependency on cloud LLM APIs, eliminating latency (typically 100-500ms for API calls) and enabling operation in connectivity-constrained environments like homes or remote locations. A production system adopting this approach would replace an expensive inference server call with a local forward pass (~10-50ms for SLMs), reducing system complexity and improving real-time responsiveness in human-robot teaming scenarios. The synthetic data augmentation technique is immediately applicable to other HRI tasks with limited labeled data—it reduces the annotation burden by allowing engineers to programmatically generate diverse interaction examples. Trade-offs include: SLMs are less general-purpose than LLMs, so domain-specific fine-tuning is often necessary; prompt engineering requires careful iteration to work well; and the performance ceiling may be lower than commercial LLM APIs on edge cases. For resource-constrained robots (mobile platforms, embedded systems), this approach reduces memory footprint and power consumption compared to running even quantized LLMs.
Limitations and When Not to Use This
The paper does not address robustness to adversarial or out-of-distribution utterances—real human communication includes sarcasm, context switches, and ambiguous role boundaries that synthetic data may not capture. The evaluation is limited to the specific HRI domain and interaction patterns in their dataset; generalization to novel environments, speaker populations, or interaction types (e.g., multilingual settings, high-stress scenarios) remains unexplored. The paper assumes that role classification is a sufficient proxy for understanding interaction dynamics, but it does not evaluate whether role predictions actually improve downstream robot behavior or human satisfaction in real deployments. Additionally, the comparison is constrained to zero-shot and one-shot settings, leaving questions about saturation performance when more labeled data is available and whether the performance gap between SLMs and LLMs widens with increased data. The work also does not address how to maintain model performance as interaction patterns evolve over time in long-running deployments.
Research Context
This paper advances the subfield of language-based HRI by questioning the implicit assumption that only large, centralized models can handle linguistic understanding in robotics. It builds on prior work in few-shot learning and prompt engineering (e.g., in-context learning) and applies these techniques to a previously under-explored problem: real-time role inference in interactive scenarios. The research aligns with broader trends in on-device AI (TinyML, edge inference) and contributes a benchmark dataset that future work can build upon. The synthetic data augmentation approach connects to curriculum learning and domain adaptation literature, offering a scalable way to bootstrap training data for specialized HRI tasks without expensive human annotation.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::