When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoning
:::info Stub — Full Engineering Breakdown Coming This paper was featured on Hugging Face Daily Papers on 2026-05-06 with 1 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Jiaqi Wei et al. |
| Year | 2026 |
| HF Upvotes | 1 |
| arXiv | 2605.03314 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
In single-stream autoregressive interfaces, the same tokens both update the model state and constitute an irreversible public commitment. This coupling creates a silence tax: additional deliberation postpones the first task-relevant content, while naive early streaming risks premature commitments that bias subsequent generations. We introduce Side-by-Side (SxS) Interleaved Reasoning, which makes disclosure timing a controllable decision within standard autoregressive generation. SxS interleaves partial disclosures with continued private reasoning in the same context, but releases content only when it is supported by the reasoning so far. To learn such pacing without incentivizing filler, we construct entailment-aligned interleaved trajectories by matching answer prefixes to supporting reasoning prefixes, then train with SFT to acquire the dual-action semantics and RL to recover reasoning performance under the new format. Across two Qwen3 architectures/scales (MoE Qwen3-30B-A3B, dense Qwen3-4B) and both in-domain (AIME25) and out-of-domain (GPQA-Diamond) benchmarks, SxS improves accuracy--content-latency Pareto trade-offs under token-level proxies such as inter-update waiting.
Engineering Breakdown
Plain English
This paper solves a fundamental tradeoff in streaming LLM outputs: either you delay showing anything while the model thinks (silence tax), or you stream early and lock in potentially wrong reasoning that biases future tokens. The authors introduce Side-by-Side Interleaved Reasoning (SxS), which lets the model maintain private reasoning while selectively releasing only the tokens it's confident about, treating disclosure timing as a learned decision rather than a binary choice.
Key Engineering Insight
The core insight is decoupling the model's internal state updates from public token commitments—you can keep reasoning privately in context while streaming only entailment-aligned content downstream. This eliminates the false choice between latency and reasoning quality by making what gets revealed a trainable policy rather than hardwired behavior.
Why It Matters for Engineers
For production systems, this directly addresses two user-facing problems: streaming latency (users see silence while models think) and hallucination propagation (early commits force models to rationalize bad outputs). If this scales, you could ship lower-latency services without sacrificing reasoning depth, which is a real tradeoff in any real-time LLM application today.
Research Context
Prior work assumed streaming and thinking had to happen in the same token sequence, forcing engineers to choose between fast-but-shallow or slow-but-thorough generation. This paper shifts the research focus from 'when should we show output' to 'what information should we reveal given what we've reasoned so far,' opening a new design space for inference-time reasoning that's hidden from users but improves output quality.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::