Neuro-Symbolic ODE Discovery with Latent Grammar Flow
| Authors | Karin Yu et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2604.16232 |
| Download | |
| Categories | cs.LG, cs.AI, cs.CE, cs.SC |
Abstract
Understanding natural and engineered systems often relies on symbolic formulations, such as differential equations, which provide interpretability and transferability beyond black-box models. We introduce Latent Grammar Flow (LGF), a neuro-symbolic generative framework for discovering ordinary differential equations from data. LGF embeds equations as grammar-based representations into a discrete latent space and forces semantically similar equations to be positioned closer together with a behavioural loss. Then, a discrete flow model guides the sampling process to recursively generate candidate equations that best fit the observed data. Domain knowledge and constraints, such as stability, can be either embedded into the rules or used as conditional predictors.
Engineering Breakdown
Plain English
This paper presents Latent Grammar Flow (LGF), a neuro-symbolic framework that discovers ordinary differential equations (ODEs) from observational data by combining neural networks with symbolic reasoning. The key innovation is embedding grammar-based equation representations into a discrete latent space where semantically similar equations cluster together, then using a discrete flow model to recursively generate and sample candidate equations that best match the observed data. The approach uniquely allows domain knowledge and physical constraints like stability to be incorporated either directly into the grammar rules or as conditional predictors during generation. This bridges the gap between black-box neural models and interpretable symbolic equations, enabling the discovery of transferable, human-readable scientific models.
Core Technical Contribution
The core novelty is the combination of three technical components: (1) encoding differential equations as discrete grammar-based representations rather than continuous vectors, (2) using a behavioral loss to position semantically equivalent equations close together in latent space despite syntactic differences, and (3) applying a discrete flow model for guided sampling that respects both data fit and domain constraints. Prior work on symbolic regression and neural ODE discovery typically relies on continuous representations or genetic algorithms; LGF introduces the insight that treating equations as discrete grammar objects allows direct incorporation of structural knowledge and semantic similarity without continuous relaxation. The behavioral loss is particularly novel—it measures whether two syntactically different equations produce similar dynamics on the data, forcing the latent space to organize by mathematical meaning rather than notation. This enables conditional generation where physics-based constraints like Lyapunov stability can steer the sampling process toward valid, interpretable solutions.
How It Works
The system operates in three stages: (1) Representation & Embedding: Raw differential equations are parsed into grammar-based abstract syntax trees (ASTs), where each equation is a sequence of grammar productions (e.g., 'equation → polynomial_term + trigonometric_term'). These discrete representations are embedded into a continuous latent space via a learned encoder, creating a structured space where equations with similar dynamics cluster. (2) Latent Space Organization: A behavioral loss function is applied during training by simulating each equation on the observed data and comparing trajectories; equations producing similar behaviors are pulled closer together in latent space, while syntactically identical equations with different dynamics are pushed apart. This creates a semantically meaningful organization. (3) Guided Generation: A discrete flow model (similar to diffusion on discrete spaces) samples from this organized latent space, starting from random noise and iteratively denoising to generate new equation candidates. At each step, the model can condition on domain constraints (stability, sparsity, physical laws) by incorporating them as signals into the flow network, ensuring generated equations satisfy required properties before evaluating them against data. The final candidate equations are ranked by fit quality and converted back to interpretable symbolic form.
Production Impact
For engineers building scientific computing systems, this approach solves the core problem of automated equation discovery with interpretability guarantees—you get human-readable differential equations rather than neural network weights. In a production ML pipeline for physics-informed machine learning, you could replace expensive manual modeling steps: instead of domain experts hand-crafting equations, LGF could propose candidates that satisfy your known constraints (e.g., conservation laws, stability requirements) and evaluate them against real sensor data or simulations. This is particularly valuable in domains like control systems, climate modeling, or engineering design where regulatory and safety requirements demand interpretable models. The trade-offs are significant: computational cost is likely higher than standard neural ODE solvers due to the discrete flow sampling and repeated simulation-based evaluations; you need sufficient trajectory data to learn the behavioral loss effectively; and the grammar rules must be carefully designed by domain experts to ensure the latent space captures relevant equation structure. Integration complexity is moderate—you'd need to define domain-specific grammars and potentially retrain the flow model for different problem classes, but inference (equation sampling) should be reasonably fast once trained.
Limitations and When Not to Use This
The paper assumes that meaningful differential equations exist in your domain and can be represented within the specified grammar—if your system dynamics are truly non-Markovian or require delay differential equations outside the grammar, LGF will fail gracefully but inefficiently. The behavioral loss requires the ability to simulate candidate equations on observed data efficiently; for high-dimensional or stiff ODE systems, this simulation cost dominates the overall computational budget and may be prohibitive. The approach assumes your observational data is sufficiently clean and densely sampled to accurately compare equation dynamics; noisy or sparse measurements could corrupt the semantic organization of latent space. The framework doesn't explicitly handle parameter identifiability or uncertainty quantification—it returns point estimates of discovered equations without credible intervals, which matters for downstream decision-making. Additionally, scaling to very long equations (high complexity) with large grammar spaces remains unexplored; the discrete flow model's sampling efficiency may degrade combinatorially with equation length.
Research Context
This work builds on two parallel research threads: neuro-symbolic AI (combining neural networks with symbolic reasoning, exemplified by works on neural-guided theorem proving and differentiable programming) and symbolic regression (genetic programming approaches like PySR that search equation spaces). It advances over prior neural ODE discovery methods by adding interpretability and over pure symbolic regression by leveraging learned latent spaces to guide the search more efficiently. The paper likely benchmarks against standard ODE discovery datasets (like those from SciML community or physics discovery benchmarks) and genetic programming baselines. The research opens directions toward incorporating causal inference into equation discovery, extending to PDEs and integro-differential equations, and developing fully differentiable discrete flow models that could backpropagate through discrete grammar spaces more efficiently.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::