How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Models
| Authors | Gregory N. Frank |
| Year | 2026 |
| HF Upvotes | 1 |
| arXiv | 2604.04385 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
This paper localizes the policy routing mechanism in alignment-trained language models. An intermediate-layer attention gate reads detected content and triggers deeper amplifier heads that boost the signal toward refusal. In smaller models the gate and amplifier are single heads; at larger scale they become bands of heads across adjacent layers. The gate contributes under 1% of output DLA, but interchange testing (p<0.001) and knockout cascade confirm it is causally necessary. Interchange screening at n>=120 detects the same motif in twelve models from six labs (2B to 72B), though specific heads differ by lab. Per-head ablation weakens up to 58x at 72B and misses gates that interchange identifies; interchange is the only reliable audit at scale. Modulating the detection-layer signal continuously controls policy from hard refusal through evasion to factual answering. On safety prompts the same intervention turns refusal into harmful guidance, showing the safety-trained capability is gated by routing rather than removed. Thresholds vary by topic and by input language, and the circuit relocates across generations within a family while behavioral benchmarks register no change. Routing is early-commitment: the gate commits at its own layer before deeper layers finish processing the input. Under an in-context substitution cipher, gate interchange necessity collapses 70 to 99% across three models and the model switches to puzzle-solving. Injecting the plaintext gate activation into the cipher forward pass restores 48% of refusals in Phi-4-mini, localizing the bypass to the routing interface. A second method, cipher contrast analysis, uses plain/cipher DLA differences to map the full cipher-sensitive routing circuit in O(3n) forward passes. Any encoding that defeats detection-layer pattern matching bypasses the policy regardless of whether deeper layers reconstruct the content.
Engineering Breakdown
Plain English
This paper identifies and characterizes a specific mechanistic component in alignment-trained language models that implements policy routing for refusal behavior. The authors locate an intermediate-layer attention gate that detects safety-relevant content and triggers amplifier heads in deeper layers to boost the refusal signal, with the gate itself contributing less than 1% to output logits but proven causally necessary through interchange testing (p<0.001). They demonstrate this motif exists across twelve models from six different labs ranging from 2B to 72B parameters, though the specific head locations vary by organization. A key finding is that interchange-based circuit detection is the only reliable method at scale—per-head ablation misses gates that interchange identifies and shows 58x weaker effects on the largest models.
Core Technical Contribution
The core contribution is a mechanistic audit method for safety mechanisms in large language models using interchange intervention testing, which reliably detects causal components that account for <1% of output logits but are structurally necessary for alignment behavior. Unlike standard ablation techniques that either destroy too much (losing the causal structure) or too little (missing distributed components), interchange testing isolates individual causal nodes by swapping activations between clean and corrupted runs and measuring the resulting perturbation. The authors discover that alignment-trained models implement refusal through a two-stage gate-and-amplifier architecture where attention heads in middle layers detect safety-relevant content and gate information flow, which is then amplified by downstream heads. This mechanistic discovery is novel because it reveals that policy routing is not a monolithic mechanism but a distributed circuit spanning multiple layers and head types, and it generalizes across model families despite architectural differences.
How It Works
The mechanism operates in three stages: first, an intermediate-layer attention gate reads and detects safety-relevant content in the token sequence; second, when content is detected, the gate reduces information flow through certain channels (contributing <1% directly to output logits but acting as a control signal); third, this gating signal modulates downstream amplifier heads in deeper layers that boost the safety token's logit, amplifying it toward refusal. In smaller models (2B scale), the gate and amplifier are individual attention heads in adjacent layers. At larger scales (72B), these components expand into bands of multiple heads across several adjacent layers, suggesting that the computational burden of safety routing is distributed to maintain efficiency. Interchange testing works by creating two forward passes: a clean pass on safe input and a corrupted pass on unsafe input; then swapping the activations of a specific head between the two runs and measuring how much the output logit changes relative to the original corrupted output. If swapping that head's activation causes the output to shift significantly toward the clean pass behavior, the head is causally involved in the safety mechanism.
Production Impact
For production systems, this research enables mechanistic auditing of black-box safety properties—you can now reliably identify which specific components are responsible for refusal behavior without requiring model source code or training details, which matters when operating third-party or externally-trained models. This allows safety teams to perform targeted testing: if you understand which heads implement the refusal gate, you can run focused red-teaming against those components specifically, design interventions (like safety fine-tuning) that target the identified circuit, or monitor their activations at inference time as a safety signal. The production workflow would involve: (1) running interchange screening at n≥120 test cases to map the gate-amplifier circuit in your model, (2) extracting the head coordinates and layer ranges, (3) instrumenting the model to log or monitor these specific activations, (4) using those signals for inference-time safety verification or as feedback for alignment training. The main trade-off is computational cost—interchange testing requires n forward passes (120+) for reliable detection, so you'd run this once at model validation time rather than continuously, and the resulting audit is model-specific, meaning each new model checkpoint requires re-screening.
Limitations and When Not to Use This
The paper does not address what happens when models are fine-tuned, quantized, or distilled after alignment training—it's unclear whether the gate-amplifier circuit remains mechanistically identical or degrades, which is critical for production systems that often apply post-training compression. The method requires running 120+ forward passes with activation swapping, which is computationally expensive and scales poorly; while interchange is more reliable than per-head ablation, there's no analysis of how detection reliability changes with very large models or vision-language models, or whether the method works on models trained with different alignment procedures beyond RLHF. The paper identifies gates in only twelve models from six labs; architectural diversity is limited (all are transformer-based), so it's unknown whether the gate-amplifier motif generalizes to mixture-of-experts models, other alignment methods (DPO, IPO, etc.), or models trained with different safety objectives like constitutional AI. Finally, the paper describes detection and localization but not intervention—knowing where the safety mechanism lives doesn't directly tell you how to make it more robust, prevent bypass attacks, or improve it, leaving the causal discovery results somewhat disconnected from actionable safety improvements.
Research Context
This work builds on mechanistic interpretability research pioneered by works on transformer circuits and causal intervention methods, extending techniques like activation patching and interchange testing to the alignment domain specifically. It directly addresses a gap in safety research: while RLHF and other alignment methods empirically reduce refusal failures, very little work has looked inside trained models to understand which components implement safety policies, making it hard to audit, debug, or improve alignment robustness. The paper's finding that alignment behavior is implemented as a distributed gate-and-amplifier circuit across layers resonates with prior discoveries that transformers implement complex multi-layer computations for other capabilities (like in-context learning), suggesting alignment may follow similar mechanistic patterns. This research opens a new direction: using mechanistic audit methods to discover and characterize safety circuits, which could enable better red-teaming (targeting the actual safety components), more interpretable alignment training (optimizing for interpretable safety mechanisms), and better understanding of why different labs' models use different head coordinates (suggesting different learning dynamics or objectives).
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::