Context-Value-Action Architecture for Value-Driven Large Language Model Agents
| Authors | TianZe Zhang et al. |
| Year | 2026 |
| HF Upvotes | 5 |
| arXiv | 2604.05939 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Large Language Models (LLMs) have shown promise in simulating human behavior, yet existing agents often exhibit behavioral rigidity, a flaw frequently masked by the self-referential bias of current "LLM-as-a-judge" evaluations. By evaluating against empirical ground truth, we reveal a counter-intuitive phenomenon: increasing the intensity of prompt-driven reasoning does not enhance fidelity but rather exacerbates value polarization, collapsing population diversity. To address this, we propose the Context-Value-Action (CVA) architecture, grounded in the Stimulus-Organism-Response (S-O-R) model and Schwartz's Theory of Basic Human Values. Unlike methods relying on self-verification, CVA decouples action generation from cognitive reasoning via a novel Value Verifier trained on authentic human data to explicitly model dynamic value activation. Experiments on CVABench, which comprises over 1.1 million real-world interaction traces, demonstrate that CVA significantly outperforms baselines. Our approach effectively mitigates polarization while offering superior behavioral fidelity and interpretability.
Engineering Breakdown
Plain English
This paper identifies a critical flaw in how LLM agents are evaluated and trained: current prompt-based reasoning approaches actually reduce behavioral diversity and increase value polarization rather than improving agent fidelity. The authors discovered that intensifying reasoning prompts doesn't help—it makes the problem worse—because existing evaluation methods rely on LLMs judging themselves (self-referential bias) rather than comparing against real human behavior. To fix this, they propose CVA (Context-Value-Action), a new architecture that separates value reasoning from action generation using a specialized Value Verifier trained on authentic human data, grounded in established psychology theories (Schwartz's value theory and the S-O-R model). Early results suggest this approach maintains population diversity while improving alignment with actual human behavioral patterns.
Core Technical Contribution
The core innovation is the Context-Value-Action architecture, which decouples action generation from cognitive reasoning—a fundamental departure from end-to-end prompt-based agents. Rather than having a single model perform reasoning and action selection jointly, CVA introduces a Value Verifier component trained on authentic human behavioral data to validate whether proposed actions align with genuine human values, not LLM-predicted values. This is grounded in psychological theory (Schwartz's Basic Human Values and Stimulus-Organism-Response model) rather than pure scaling or prompt engineering. The key insight is that value alignment requires learning from real human ground truth, not from self-verification loops that amplify LLM biases and create artificial behavioral consensus.
How It Works
The CVA architecture operates as a pipeline: (1) Context Input receives the agent's situational state and constraints; (2) Value Extraction analyzes what human values are relevant to the current context using Schwartz's value taxonomy; (3) Action Generation produces candidate actions; (4) Value Verifier—the critical component—validates each candidate action against learned authentic human value-action mappings, filtering out actions that don't align with real human behavior patterns. Unlike traditional approaches where the LLM reasons internally and produces actions, the Value Verifier is a separate model trained on empirical human data that acts as a gating mechanism. The S-O-R (Stimulus-Organism-Response) model provides the theoretical framework: stimulus is context, organism is the value reasoning system, and response is the verified action. This decoupling prevents the feedback collapse that occurs when LLMs reinforce their own reasoning patterns.
Production Impact
Adopting CVA would fundamentally change how you build and evaluate LLM agents: instead of relying on prompt engineering and self-verification (which are cheap but biased), you'd need to invest in collecting and curating authentic human behavioral data for training the Value Verifier component. This increases upfront data costs but eliminates the silent failure mode where agents appear good in automated benchmarks but diverge from real human behavior in production. The architectural separation of values from actions also enables better interpretability and control—you can audit why specific actions were rejected without unwinding a monolithic model's reasoning. Latency impact is moderate: you add a forward pass through the Value Verifier for each action candidate, but this is parallelizable and doesn't require expensive reasoning chains. Integration complexity is higher than baseline LLM approaches because you need (a) human behavioral datasets, (b) a separate Value Verifier model with its own infrastructure, and (c) mechanisms to update value mappings as real-world behavior patterns shift.
Limitations and When Not to Use This
The paper's reliance on Schwartz's value theory assumes that human values can be meaningfully discretized into a fixed taxonomy—this may not capture context-dependent or cultural variations in what people actually value. The approach requires high-quality ground truth human behavioral data to train the Value Verifier, which is expensive to collect at scale and may encode biases from the specific human populations sampled. The paper doesn't clearly specify how the Value Verifier handles edge cases or novel action-value combinations it hasn't seen during training, potentially limiting generalization to truly novel situations. Additionally, while the architecture avoids prompt-injection and reasoning collapse, it introduces a new failure mode: a poorly trained Value Verifier could systematically reject valid actions, creating behavioral brittleness. The paper also doesn't address computational trade-offs compared to smaller, specialized models that might achieve similar diversity with lower overhead.
Research Context
This work builds on a growing recognition that 'LLM-as-judge' evaluation is fundamentally flawed due to self-referential bias—a concern also raised in recent work on jailbreaking and reward hacking in RLHF systems. It extends psychological theories (Schwartz's Basic Human Values, S-O-R model) into the LLM domain, drawing parallel to human behavioral science rather than relying purely on information-theoretic approaches. The emphasis on empirical ground truth evaluation against synthetic benchmarks echoes methodological critiques of existing agent benchmarks, positioning CVA as part of a broader shift toward real-world behavioral validation. The decoupling of reasoning from action generation relates to modular agent architectures and tool-use frameworks, but with a focus on value alignment as the critical modularity boundary rather than functional task separation.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::