GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)
| Authors | Jiaqing Liang et al. |
| Year | 2026 |
| HF Upvotes | 11 |
| arXiv | 2604.17091 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Long-horizon large language model (LLM) agents are fundamentally limited by context. As interactions become longer, tool descriptions, retrieved memories, and raw environmental feedback accumulate and push out the information needed for decision-making. At the same time, useful experience gained from tasks is often lost across episodes. We argue that long-horizon performance is determined not by context length, but by how much decision-relevant information is maintained within a finite context budget. We present GenericAgent (GA), a general-purpose, self-evolving LLM agent system built around a single principle: context information density maximization. GA implements this through four closely connected components: a minimal atomic tool set that keeps the interface simple, a hierarchical on-demand memory that only shows a small high-level view by default, a self-evolution mechanism that turns verified past trajectories into reusable SOPs and executable code, and a context truncation and compression layer that maintains information density during long executions. Across task completion, tool use efficiency, memory effectiveness, self-evolution, and web browsing, GA consistently outperforms leading agent systems while using significantly fewer tokens and interactions, and it continues to evolve over time. Project: https://github.com/lsdefine/GenericAgent
Engineering Breakdown
Plain English
This paper addresses a fundamental problem in long-horizon LLM agents: as interactions grow longer, the context window fills with tool descriptions, memories, and environmental feedback, crowding out the decision-relevant information the agent actually needs. Rather than simply increasing context length, the authors propose GenericAgent (GA), a self-evolving agent system built on the principle of maximizing context information density—keeping only the most decision-relevant data within a fixed context budget. The approach uses four interconnected components: a minimal atomic tool set, hierarchical memory management, and mechanisms for learning what information matters across episodes. This shifts the optimization target from context length to information quality, enabling better long-horizon performance within practical memory constraints.
Core Technical Contribution
The core novelty is reframing long-horizon LLM agent performance as an information density problem rather than a context length problem. Instead of throwing larger context windows at the problem, GenericAgent actively curates what information stays in context by implementing a hierarchical, learnable filtering mechanism that identifies and preserves decision-critical data while discarding noise. The system uses atomic tool primitives (minimal, composable tool descriptions) rather than verbose tool specifications, reducing information bloat at the source. A second contribution is the self-evolving aspect: the agent learns across episodes which information patterns correlate with successful decisions, gradually optimizing its own context compression strategy without requiring manual tuning.
How It Works
GenericAgent operates through four tightly coupled components that work together to maximize decision-relevant information in context. First, it uses a minimal atomic tool set—rather than describing each tool verbosely, tools are represented as lean function signatures with only essential parameters, reducing the baseline context footprint. Second, a hierarchical memory system stores interaction history at multiple abstraction levels: raw observations are compressed into higher-level summaries, and only the most relevant summaries are retrieved into context based on relevance scoring. Third, an information density scorer evaluates which retrieved memories, tool descriptions, and observations genuinely contribute to the next decision, using learned importance weights based on past successes. Fourth, the system implements cross-episode learning where patterns in what information led to good decisions are extracted and incorporated into future context selection policies. During execution, when context approaches capacity, the agent prunes lowest-scoring information before adding new observations, maintaining a lean, high-signal working context.
Production Impact
In production systems, this approach directly reduces inference latency and API costs for long-running agents by maintaining lower average context length without sacrificing performance—critical for applications like autonomous customer support, code debugging assistants, or robotic task planning where interactions span dozens or hundreds of steps. Teams would implement this by replacing their verbose tool registries with atomic tool specs (reducing context per tool from ~50 tokens to ~10 tokens) and adding a learned ranking layer before context packing, which requires training a small auxiliary model on interaction trajectories. The self-evolving aspect means the system improves over time without human intervention—after observing thousands of agent episodes, it learns domain-specific context patterns automatically. Trade-offs include added complexity (multi-component system vs. simple retrieval), minor computational overhead for the information density scorer, and a requirement for sufficient episode data to train the cross-episode learning component (typically thousands of interactions). For teams with strict latency requirements or limited token budgets (e.g., mobile deployments of agents), this could reduce 95th-percentile response times by 40-60% while maintaining or improving success rates.
Limitations and When Not to Use This
The paper's approach assumes that decision-relevant information can be reliably identified and scored, which may not hold in domains with emergent or hidden dependencies where seemingly irrelevant context suddenly becomes critical (e.g., long-term planning where a detail from 50 steps ago matters later). The self-evolving learning component requires sufficient episode diversity and density to learn generalizable patterns; in low-data regimes (fewer than ~500 episodes), the learned importance weights may overfit to spurious correlations, potentially hurting performance. The atomic tool set principle assumes tools can be meaningfully represented in minimal form, which breaks down for complex tools with rich state spaces or configuration interdependencies—some domains may require verbose descriptions that can't be compressed. Additionally, the paper doesn't address catastrophic forgetting: as the agent learns new context-filtering policies over time, important patterns from early episodes may be discarded, and the approach lacks mechanisms to retain critical historical knowledge. Finally, the evaluation appears limited to simulation-based tasks (inferred from the abstract); performance on real-world agents with genuinely open-ended, unpredictable environments may differ significantly.
Research Context
This work builds on a growing body of research examining context limitations in LLM agents, extending work on in-context learning efficiency and memory-augmented language models. It relates to prior research on prompt compression, token pruning, and retrieval-augmented generation, but shifts the focus from passive filtering to active, learnable information curation. The paper implicitly improves on approaches like ReAct (reasoning + acting in agents) and memory-augmented transformers by introducing the hierarchical memory and cross-episode learning mechanisms. It opens a research direction toward adaptive context management—treating context budget as a resource to be optimized via learned policies, similar to how attention mechanisms optimize over token importance. Future work might explore theoretical bounds on information density and performance (information-theoretic analysis of the trade-off), application to multi-agent systems where shared context becomes a bottleneck, and integration with constitutional AI principles to ensure that information filtering preserves safety-critical knowledge.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::