Neural Computers
| Authors | Mingchen Zhuge et al. |
| Year | 2026 |
| HF Upvotes | 24 |
| arXiv | 2604.06425 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
We propose a new frontier: Neural Computers (NCs) -- an emerging machine form that unifies computation, memory, and I/O in a learned runtime state. Unlike conventional computers, which execute explicit programs, agents, which act over external execution environments, and world models, which learn environment dynamics, NCs aim to make the model itself the running computer. Our long-term goal is the Completely Neural Computer (CNC): the mature, general-purpose realization of this emerging machine form, with stable execution, explicit reprogramming, and durable capability reuse. As an initial step, we study whether early NC primitives can be learned solely from collected I/O traces, without instrumented program state. Concretely, we instantiate NCs as video models that roll out screen frames from instructions, pixels, and user actions (when available) in CLI and GUI settings. These implementations show that learned runtimes can acquire early interface primitives, especially I/O alignment and short-horizon control, while routine reuse, controlled updates, and symbolic stability remain open. We outline a roadmap toward CNCs around these challenges. If overcome, CNCs could establish a new computing paradigm beyond today's agents, world models, and conventional computers.
Engineering Breakdown
Plain English
This paper introduces Neural Computers (NCs), a new machine paradigm that treats a learned model as a complete computing system rather than just a learned function or agent operating in an external environment. The authors propose that NCs unify computation, memory, and I/O as learned runtime state, eliminating the need for explicit programs or external execution environments. As an initial step, they demonstrate whether NC primitives can be learned from I/O traces alone (screen frames) without instrumented program state, instantiating NCs as video models that can roll out predicted future frames. The work is foundational research toward Completely Neural Computers (CNCs)—general-purpose systems with stable execution, explicit reprogramming, and reusable learned capabilities.
Core Technical Contribution
The core novelty is reconceptualizing a trained model as a functioning computer rather than as an agent or world model operating separately from computation. Instead of executing explicit code (conventional computers), mapping external actions to environment states (agents), or learning environment dynamics passively (world models), NCs embed computation, memory management, and I/O handling directly into learned model weights and activations. The key technical insight is that video prediction models—which must implicitly learn screen layouts, state persistence, and interaction effects—can be viewed as learning an internal representation of computational runtime, effectively learning what a computer's execution stack should be. This shifts the abstraction from 'how do I predict the next frame' to 'I am learning how to be the computer that generates coherent program execution traces.'
How It Works
The approach treats a trained video model as a Neural Computer by feeding it sequences of screen frames (observations of a running computer system) and training it to predict the next frame in that sequence. The model learns to implicitly capture the system state—what programs are running, memory contents, I/O buffers—encoded in visual patterns rather than explicit state vectors. During inference, the NC rolls out predictions autoregressively: given current frame(s), predict the next frame(s), then use those predictions as input to predict further frames, creating a simulation of a running system. The key architectural component is a video diffusion or autoregressive transformer model that learns the dynamics of how system state visibly changes frame-to-frame, without ever seeing explicit program instructions or CPU state. The learned model acts as both the compute engine and the state repository—all information about what the system should do next is encoded in the model weights and the current visual input.
Production Impact
This work has significant long-term implications but is currently foundational research rather than immediately production-ready. If successfully matured, Neural Computers could enable building general-purpose systems without explicit programming languages or OS kernels—the model itself becomes the runtime, potentially simplifying deployment of AI systems that must maintain internal state and handle complex interactions. For near-term engineering, this suggests exploring whether video models trained on system traces could emulate or predict software behavior for testing, monitoring, or anomaly detection without reverse-engineering explicit code. The trade-offs are substantial: training such models requires large volumes of diverse execution traces (high data cost), inference latency depends on frame generation speed (likely slower than native code execution), and determinism is not guaranteed (learned models are stochastic). Integration into production would likely begin as a simulation/testing tool rather than replacing actual execution, and would require solving challenges around reproducibility, security boundaries, and verification.
Limitations and When Not to Use This
The paper does not address how to ensure deterministic or verifiable execution—a critical gap for production systems where correctness and reproducibility are non-negotiable. The approach assumes that I/O traces alone contain sufficient information to learn system semantics without explicit instruction state, but complex concurrent programs, memory corruption, or subtle race conditions may not leave visible signatures on screen frames. Scalability remains unproven: training on entire operating systems (Linux, Windows) with millions of possible states and branching execution paths would require architectural innovations and vastly more data than current models. The paper does not provide baselines on standard benchmarks, quantitative evaluation metrics, or comparisons to alternatives, leaving open questions about whether this approach actually learns robust computational semantics or merely high-quality video prediction.
Research Context
This work sits at the intersection of world models, neurosymbolic AI, and program synthesis research. It builds conceptually on recent advances in scaling video diffusion models (like Sora, Gemini 2.0 Video) that learn to generate long, coherent visual sequences, but applies them to a new domain: system execution traces. The work extends prior research on learning simulators of complex environments (e.g., planning with learned models, neural process models) by proposing that the simulator itself becomes the programming substrate. It opens a new research direction asking whether we can learn entire software systems as learned functions—challenging classical assumptions that algorithms must be explicit and execution must be deterministic. This could influence future work in few-shot program learning, automated code generation, and AI systems that must maintain and reason about internal computational state without human-written code.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::