General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks
| Authors | Junlin Liu et al. |
| Year | 2026 |
| HF Upvotes | 8 |
| arXiv | 2604.11778 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Contemporary large language models (LLMs) have demonstrated remarkable reasoning capabilities, particularly in specialized domains like mathematics and physics. However, their ability to generalize these reasoning skills to more general and broader contexts--often termed general reasoning--remains under-explored. Unlike domain-specific reasoning, general reasoning relies less on expert knowledge but still presents formidable reasoning challenges, such as complex constraints, nested logical branches, and semantic interference. To address this gap, we introduce General365, a benchmark specifically designed to assess general reasoning in LLMs. By restricting background knowledge to a K-12 level, General365 explicitly decouples reasoning from specialized expertise. The benchmark comprises 365 seed problems and 1,095 variant problems across eight categories, ensuring both high difficulty and diversity. Evaluations across 26 leading LLMs reveal that even the top-performing model achieves only 62.8% accuracy, in stark contrast to the near-perfect performances of LLMs in math and physics benchmarks. These results suggest that the reasoning abilities of current LLMs are heavily domain-dependent, leaving significant room for improvement in broader applications. We envision General365 as a catalyst for advancing LLM reasoning beyond domain-specific tasks toward robust, general-purpose real-world scenarios. Code, Dataset, and Leaderboard: https://general365.github.io
Engineering Breakdown
Plain English
This paper introduces General365, a new benchmark for evaluating how well large language models can perform general reasoning tasks that don't require specialized domain expertise. The authors identified a gap in existing LLM evaluation—while models excel at domain-specific reasoning (math, physics), their ability to handle general reasoning with complex constraints and logical branching remains poorly understood. General365 restricts background knowledge to K-12 level to isolate pure reasoning capability from specialized expertise, containing 365 carefully designed problems that test semantic understanding, nested logical structures, and constraint satisfaction without requiring expert knowledge.
Core Technical Contribution
The core contribution is the General365 benchmark itself, which decouples general reasoning ability from domain-specific knowledge by design. Unlike existing benchmarks that test mathematical reasoning (GSM8K), coding (HumanEval), or physics, this benchmark explicitly constrains problems to K-12 curriculum level while maintaining high reasoning complexity through semantic interference, multiple nested logical branches, and constraint satisfaction challenges. The authors created a systematic methodology for generating reasoning problems that isolate the reasoning capability itself rather than reliance on specialized training data or expert knowledge, filling a critical gap in LLM evaluation where reasoning and knowledge were previously conflated.
How It Works
General365 works by presenting language models with reasoning problems constrained to K-12 level background knowledge while introducing reasoning difficulty through structural and semantic complexity. Each problem in the benchmark is designed to require multiple steps of logical inference, handle conflicting or ambiguous constraints, and navigate semantic distractors that don't provide additional information but compete for model attention. The benchmark evaluates model outputs against ground-truth answers, measuring both correctness and potentially intermediate reasoning steps to understand where models fail. By fixing the knowledge domain (K-12) as a constant, the benchmark isolates reasoning performance from knowledge acquisition, making it possible to definitively measure whether reasoning improvements come from better logical processing or simply from better knowledge coverage.
Production Impact
For production systems, General365 provides engineers with a concrete way to evaluate whether their LLMs have genuine reasoning capabilities beyond pattern matching and memorization. This is critical for applications like customer support automation, documentation Q&A, and logical decision-making where the model must handle novel constraint combinations that differ from training data. Adopting this benchmark in your evaluation pipeline would let you distinguish between models that appear intelligent due to broad knowledge versus models that actually perform complex logical reasoning—a distinction that directly affects reliability for safety-critical applications. The trade-off is that K-12 level benchmarks don't assess domain-specific reasoning performance, so you'd need to run General365 alongside domain-specific benchmarks (like GSM8K for math) to get a complete picture of model capabilities.
Limitations and When Not to Use This
The paper's scope is limited to K-12 level problems, which may not reflect the complexity and knowledge requirements of real-world enterprise applications where domain expertise is often essential. The benchmark assumes that general reasoning can be meaningfully separated from specialized knowledge, but in practice many real reasoning tasks require integrating domain-specific facts with logical inference—the 365 problems may not capture this interplay. The paper doesn't discuss potential biases in problem generation, cultural specificity of K-12 curricula across regions, or how the benchmark handles ambiguous problems where multiple reasoning paths lead to different valid answers. Additionally, the abstract doesn't mention how the benchmark scales to more complex reasoning chains or whether performance on General365 actually predicts real-world reasoning capabilities in production systems.
Research Context
This work builds on a growing recognition that benchmarking LLMs requires going beyond task-specific metrics to understand fundamental capabilities. Prior work established domain-specific reasoning benchmarks (GSM8K for math word problems, ScienceQA for science reasoning), but General365 addresses the understudied problem of general reasoning that doesn't rely on specialized expertise. The paper contributes to the broader research direction of decomposing LLM capabilities into interpretable components—separating knowledge from reasoning, allowing clearer diagnosis of model strengths and weaknesses. This benchmark enables future research into training methods that improve general reasoning specifically, rather than capabilities that conflate reasoning with knowledge accumulation.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::