Red Queen: Exposing Latent Multi-Turn Risks in Large Language Models.
| Authors | Yifan Jiang 0001 et al. |
| Year | 2025 |
| Venue | ACL 2025 |
| Paper | View on ACL Anthology |
| PDF | Download |
Abstract
Abstract not yet available in this stub. Read the full paper →
Engineering Breakdown
Plain English
Red Queen investigates latent risks that emerge in multi-turn conversations with large language models—risks that don't appear in single-turn interactions. The paper exposes how LLMs can gradually shift behavior, accumulate context in problematic ways, or be steered toward harmful outputs across multiple dialogue turns without triggering safety mechanisms designed for isolated queries. This is a critical gap because production LLM systems handle conversation threads where early turns influence later ones, yet most safety evaluation focuses on single isolated prompts. The authors introduce a systematic framework to identify and measure these multi-turn risks, demonstrating concrete attack vectors that evade standard defenses.
Core Technical Contribution
The core contribution is identifying and formalizing multi-turn jailbreak attacks that exploit temporal dependencies in conversation context—a failure mode that doesn't exist or isn't visible in single-turn evaluation. The paper introduces Red Queen, a methodology for systematically discovering how safety guardrails can be circumvented through conversational trajectory rather than isolated prompts. This is fundamentally different from prior work that tests LLMs against individual adversarial inputs; instead it models attack as a sequence where each turn conditions on previous context, allowing attackers to establish rapport, build assumptions, or gradually normalize harmful requests. The technical novelty lies in formalizing this as a search problem over conversation trajectories and demonstrating that standard RLHF-trained safety mechanisms are vulnerable to these sequential attacks.
How It Works
The Red Queen framework operates by generating multi-turn dialogue sequences where the adversary strategically selects prompts across conversation history to reach a target harmful output. The system maintains conversational context—the full dialogue history including model responses and user prompts—as the state space, and searches over possible prompt sequences to find trajectories that elicit harmful outputs while bypassing safety checks. Each turn is conditioned on the full prior context, meaning earlier turns set up assumptions, establish false framings, or build toward the target. The framework likely uses gradient-based search or reinforcement learning to optimize the sequence of prompts, testing against target models to discover which conversation paths succeed. The key insight is that safety classifiers trained on isolated examples fail because they evaluate each turn independently, missing the cumulative effect of context manipulation across turns.
Production Impact
For production LLM systems, this research demands fundamental changes to safety evaluation and deployment strategy. Current guardrails that score individual outputs for toxicity or harmfulness are insufficient—you must implement stateful safety checks that evaluate entire conversation contexts, not just the latest user message. This means building safety layers that track conversation history, detect context drift, and reset or warn when conversation patterns become suspicious (similar to fraud detection in financial systems). The practical cost is increased latency for multi-turn interactions (each message must be evaluated against full context history rather than standalone), higher memory per conversation session, and significantly more complex safety model training. Teams building conversational AI must move from single-turn safety metrics (like the ones used in HELM or TruthfulQA benchmarks) to multi-turn red-team evaluation, which is substantially more expensive to execute but necessary for production safety.
Limitations and When Not to Use This
Red Queen assumes that safety mechanisms trained via RLHF are the primary defense, but doesn't fully address newer approaches like constitutional AI or adversarial training on multi-turn data. The paper may not generalize across all model architectures and training regimes—models with different context window sizes, instruction-tuning approaches, or safety training methods could have different vulnerability profiles. A critical gap is the computational cost of discovering these attacks; while the paper identifies the vulnerabilities, it doesn't solve the engineering problem of efficiently integrating continuous multi-turn adversarial evaluation into production testing pipelines. The work also doesn't fully explore mitigation strategies beyond detecting attacks after they occur—it's stronger on diagnosis than on treatment, leaving open the question of how to make models inherently robust to multi-turn manipulation rather than just catching it in post.
Research Context
This work builds on the growing body of adversarial attack research for LLMs (following papers on jailbreaking, prompt injection, and indirect attacks) but extends the threat model from single-turn to multi-turn interactions, which is closer to real deployment scenarios. It relates to safety evaluation frameworks like HELM and EleutherAI's TruthfulQA, but identifies a critical evaluation gap they don't cover. The paper contributes to the broader safety research direction around context-dependent vulnerabilities and temporal attack surfaces in language models, opening up new research into whether standard RLHF and constitutional AI approaches can be enhanced with multi-turn robustness. It also informs the design of production safety systems, moving the field toward conversation-level rather than message-level safety models.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::