The Cognitive Penalty: Ablating System 1 and System 2 Reasoning in Edge-Native SLMs for Decentralized Consensus
| Authors | Syed Muhammad Aqdas Rizvi |
| Year | 2026 |
| HF Upvotes | 1 |
| arXiv | 2604.16913 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Decentralized Autonomous Organizations (DAOs) are inclined explore Small Language Models (SLMs) as edge-native constitutional firewalls to vet proposals and mitigate semantic social engineering. While scaling inference-time compute (System 2) enhances formal logic, its efficacy in highly adversarial, cryptoeconomic governance environments remains underexplored. To address this, we introduce Sentinel-Bench, an 840-inference empirical framework executing a strict intra-model ablation on Qwen-3.5-9B. By toggling latent reasoning across frozen weights, we isolate the impact of inference-time compute against an adversarial Optimism DAO dataset. Our findings reveal a severe compute-accuracy inversion. The autoregressive baseline (System 1) achieved 100% adversarial robustness, 100% juridical consistency, and state finality in under 13 seconds. Conversely, System 2 reasoning introduced catastrophic instability, fundamentally driven by a 26.7% Reasoning Non-Convergence (cognitive collapse) rate. This collapse degraded trial-to-trial consensus stability to 72.6% and imposed a 17x latency overhead, introducing critical vulnerabilities to Governance Extractable Value (GEV) and hardware centralization. While rare (1.5% of adversarial trials), we empirically captured "Reasoning-Induced Sycophancy," where the model generated significantly longer internal monologues (averaging 25,750 characters) to rationalize failing the adversarial trap. We conclude that for edge-native SLMs operating under Byzantine Fault Tolerance (BFT) constraints, System 1 parameterized intuition is structurally and economically superior to System 2 iterative deliberation for decentralized consensus. Code and Dataset: https://github.com/smarizvi110/sentinel-bench
Engineering Breakdown
Plain English
This paper investigates whether scaling inference-time compute (System 2 reasoning) actually improves the robustness of small language models when used as governance firewalls in decentralized autonomous organizations (DAOs). The authors built Sentinel-Bench, an 840-inference empirical framework that systematically ablates reasoning capabilities in Qwen-3.5-9B by toggling latent computation while keeping model weights frozen, testing against adversarial proposals from Optimism DAO. The shocking finding: the simple autoregressive baseline (System 1, no extra reasoning) achieved 100% adversarial robustness and 100% juridical conformance, while scaling inference-time compute appears to degrade these metrics—a direct inversion of the expected compute-accuracy relationship.
Core Technical Contribution
The core novelty is the discovery of a compute-accuracy inversion in adversarial cryptoeconomic governance contexts—the paper demonstrates that more inference-time reasoning does not necessarily improve model robustness against semantic social engineering attacks in DAO proposal vetting. Rather than proposing a new architecture, the contribution is empirical and counterintuitive: the authors show via controlled ablation that frozen-weight System 1 baselines may outperform expensive System 2 (chain-of-thought, multi-step reasoning) approaches in this specific adversarial domain. This challenges the prevailing assumption that scaling reasoning always improves model safety and decision quality, particularly in high-stakes governance where semantic manipulation is the primary threat vector.
How It Works
The paper conducts a strict intra-model ablation on Qwen-3.5-9B: the model weights remain completely frozen across all experiments, ensuring fair comparison. The key toggle mechanism switches latent reasoning pathways on and off—this means System 1 (direct, single-pass autoregressive generation) runs as a baseline, while System 2 variants introduce multi-step reasoning (likely chain-of-thought or similar mechanisms that extend inference-time compute without retraining). Each of the 840 inferences evaluates a proposal against two metrics: adversarial robustness (ability to reject semantically engineered proposals that try to exploit governance rules) and juridical conformance (adherence to constitutional rules of the DAO). The framework feeds adversarial proposals from Optimism DAO—real or synthetic attack cases—into both System 1 and System 2 variants, measuring which reasoning depth produces correct rejection or acceptance decisions.
Production Impact
For teams deploying language models as governance filters or safety gates, this paper suggests a critical cost-benefit recalibration: inference-time compute scaling (which adds 5-50x latency and compute cost per query) may be unnecessary or even harmful in adversarial governance contexts. Instead of implementing expensive multi-step reasoning pipelines, teams might achieve better results with lean, direct-inference classifiers that are harder to confuse via semantic manipulation. In production DAO governance systems, this could mean deploying a 9B model with single-pass inference (~100ms latency, minimal compute) rather than a larger model with chain-of-thought reasoning (~5-10s latency, 50x compute). However, the trade-off is context-dependent: if your governance environment has well-formed, non-adversarial proposals, reasoning depth might still help; if you face sophisticated semantic attacks, this paper suggests you should invest in adversarial training and simpler architectures instead.
Limitations and When Not to Use This
The paper is limited to a single model architecture (Qwen-3.5-9B at 9 billion parameters) and a single DAO dataset (Optimism), raising questions about generalization to other small models, larger models, or different governance domains. The ablation methodology freezes weights, which is a clean experimental design but doesn't explore whether retraining System 2 variants (e.g., via RLHF or constitutional AI) might recover the lost robustness—the paper only compares inference-time toggles on a static checkpoint. The paper provides no analysis of why the compute-accuracy inversion occurs: is it because chain-of-thought introduces spurious reasoning paths that an adversary can exploit? Does it reduce decision confidence? Without mechanistic explanation, it's unclear how broadly this finding applies or how to predict when it will hold. Finally, 100% baseline robustness is suspiciously high and suggests the adversarial dataset may be limited in sophistication or that the juridical conformance metric may not capture all failure modes.
Research Context
This paper builds on a decade of work showing that larger-scale inference-time reasoning (chain-of-thought, tree-of-thought, process reward models) generally improves model accuracy on complex tasks, but challenges that paradigm in the specific context of adversarial governance. It intersects three research areas: (1) small language models and edge inference, which have grown important as organizations want to run models locally without cloud latency; (2) constitutional AI and AI alignment, which explores how to embed rules and values into model behavior; (3) formal verification and semantic robustness, which studies whether models can be fooled by adversarial inputs that are grammatically valid but semantically hostile. The work opens a new research direction in adversarial inference-time compute: under what conditions do more reasoning steps help vs. hurt model safety? This could influence future work on reasoning-based safety mechanisms, DAO governance infrastructure, and the cost-benefit calculus of system 2 compute in production.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::