Protecting Language Models Against Unauthorized Distillation through Trace Rewriting
| Authors | Xinhang Ma et al. |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2602.15143 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Knowledge distillation is a widely adopted technique for transferring capabilities from LLMs to smaller, more efficient student models. However, unauthorized use of knowledge distillation takes unfair advantage of the considerable effort and cost put into developing frontier models. We investigate methods for modifying teacher-generated reasoning traces to achieve two objectives that deter unauthorized distillation: (1) anti-distillation, or degrading the training usefulness of query responses, and (2) API watermarking, which embeds verifiable signatures in student models. We introduce several approaches for dynamically rewriting a teacher's reasoning outputs while preserving answer correctness and semantic coherence. Two of these leverage the rewriting capabilities of LLMs, while others use gradient-based techniques. Our experiments show that a simple instruction-based rewriting approach achieves a strong anti-distillation effect while maintaining or even improving teacher performance. Furthermore, we show that our rewriting approach also enables embedding watermarks that can be reliably detected with essentially no false alarms. Our code is available at https://github.com/xhOwenMa/trace-rewriting.
Engineering Breakdown
Plain English
This paper addresses a critical security problem: large language model developers want to prevent competitors from stealing their models through unauthorized knowledge distillation. The authors propose methods to modify the reasoning traces (step-by-step outputs) that teachers generate, making them less useful for training student models while keeping the final answers correct. They introduce two key defenses: anti-distillation (degrading training usefulness) and API watermarking (embedding verifiable signatures into student models). The core innovation is dynamically rewriting reasoning outputs using both LLM-based approaches and other techniques, preserving correctness and semantic meaning while protecting intellectual property.
Core Technical Contribution
The paper's central novelty is a framework for adversarial modification of teacher model outputs that simultaneously achieves two goals: reducing distillation effectiveness and embedding watermarks. Rather than blocking distillation entirely, the authors preserve answer correctness while corrupting the reasoning traces that make distillation valuable—this is fundamentally different from prior defenses that either block APIs or degrade all outputs indiscriminately. They present multiple rewriting mechanisms, including LLM-based approaches that leverage the teacher's own capabilities, enabling dynamic adaptation to different attack scenarios. This represents the first systematic treatment of protecting reasoning outputs specifically, not just final answers, against knowledge distillation theft.
How It Works
The system operates in three stages: first, the teacher model generates both a final answer and detailed reasoning traces (chain-of-thought or similar formats). Second, a dynamic rewriting module transforms these reasoning traces using one of several techniques—some leverage the LLM's own editing capabilities, others apply algorithmic transformations. The rewriting is constrained by two requirements: it must not change the correctness of the final answer, and it must maintain semantic coherence so the output looks natural to API users. Third, watermarking information is embedded into the rewritten traces using techniques that survive the distillation process, allowing creators to verify if a student model was trained on stolen data. The key technical insight is that reasoning traces are lower-priority for end users (who care mainly about answers) but high-value for attackers (who use traces to train student models), creating an asymmetry the authors exploit.
Production Impact
For API providers protecting frontier models, this approach offers a practical middle ground: serve functional responses to legitimate users while degrading value for distillers, without requiring output blocking or answer degradation. Implementation requires adding a rewriting module to the inference pipeline, adding roughly 5-15% latency overhead depending on the rewriting technique chosen (LLM-based approaches are more expensive than algorithmic ones). Integration complexity is moderate—you need to wrap teacher outputs and optionally implement watermark verification during student model evaluation. The concrete benefit is that unauthorized distilled models will either produce visibly degraded reasoning or fail watermark verification, creating legal/technical evidence of theft. Trade-off: legitimate users studying your reasoning will also see modified traces, potentially reducing their educational value, though final answers remain correct.
Limitations and When Not to Use This
The paper assumes attackers only use API access and don't have weight-space access to the original model, which doesn't protect against model weights being leaked or stolen directly. Anti-distillation degradation may be detectable or circumventable by sophisticated attackers who recognize the rewriting patterns or use techniques like fine-tuning to restore reasoning quality. The watermarking approach requires the attacker to preserve the embedded signatures through distillation, which may fail if they apply aggressive quantization, pruning, or continued pre-training. The paper doesn't deeply explore the semantic quality loss from rewriting—while answers stay correct, the modified reasoning traces might be incoherent or confusing to legitimate users, creating a usability penalty that's not fully quantified. Follow-up work needs to address robustness against adaptive attacks where adversaries explicitly try to remove watermarks or reverse-engineer the rewriting rules.
Research Context
This work builds on the knowledge distillation security literature, extending prior defenses that focus on blocking access or adding noise to all outputs. It connects to watermarking research in ML (model fingerprinting, backdoor-based watermarks) by applying watermarking to reasoning traces rather than weights. The paper addresses a gap in LLM IP protection: while prior work targets model theft via weight extraction or fine-tuning, this work specifically defends against capability stealing via distillation, which is cheaper and more accessible to competitors. It sits at the intersection of safety (protecting legitimate IP), applied ML (production API defense), and adversarial robustness (defending against sophisticated distillation attacks), opening a research direction on output-level defenses for large models.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::