Heterogeneous Scientific Foundation Model Collaboration
| Authors | Zihao Li et al. |
| Year | 2026 |
| HF Upvotes | 193 |
| arXiv | 2604.27351 |
| Download | |
| Code | https://github.com/Violet24K/Eywa |
Abstract
Agentic large language model systems have demonstrated strong capabilities. However, their reliance on language as the universal interface fundamentally limits their applicability to many real-world problems, especially in scientific domains where domain-specific foundation models have been developed to address specialized tasks beyond natural language. In this work, we introduce Eywa, a heterogeneous agentic framework designed to extend language-centric systems to a broader class of scientific foundation models. The key idea of Eywa is to augment domain-specific foundation models with a language-model-based reasoning interface, enabling language models to guide inference over non-linguistic data modalities. This design allows predictive foundation models, which are typically optimized for specialized data and tasks, to participate in higher-level reasoning and decision-making processes within agentic systems. Eywa can serve as a drop-in replacement for a single-agent pipeline (EywaAgent) or be integrated into existing multi-agent systems by replacing traditional agents with specialized agents (EywaMAS). We further investigate a planning-based orchestration framework in which a planner dynamically coordinates traditional agents and Eywa agents to solve complex tasks across heterogeneous data modalities (EywaOrchestra). We evaluate Eywa across a diverse set of scientific domains spanning physical, life, and social sciences. Experimental results demonstrate that Eywa improves performance on tasks involving structured and domain-specific data, while reducing reliance on language-based reasoning through effective collaboration with specialized foundation models.
Engineering Breakdown
Plain English
Eywa is a framework that solves a fundamental limitation of current AI systems: language models can only reason about text, but real scientific problems require reasoning over specialized data like molecular structures, protein sequences, or astronomical data. The authors introduce a heterogeneous agentic system where a language model acts as a reasoning interface that coordinates multiple domain-specific foundation models optimized for non-linguistic modalities. This allows LLM-based agents to orchestrate inference across scientific foundation models (e.g., protein folding models, molecular property predictors) by learning to compose their outputs and guide their execution, extending agentic AI beyond text-only tasks to complex scientific domains.
Core Technical Contribution
The key innovation is augmenting domain-specific foundation models with a language-model-based reasoning interface that enables cross-modal orchestration without requiring the domain models to be modified. Rather than forcing scientific tasks into a text representation, Eywa keeps domain models in their native modalities while the LLM learns to route queries, interpret outputs, and coordinate multi-model inference chains. This is fundamentally different from prior work that either treats all modalities as text embeddings or builds monolithic multimodal models—instead, Eywa treats heterogeneous foundation models as tools that an LLM agent can invoke, learn from, and compose. The approach preserves the specialized optimization of domain models while leveraging the reasoning and planning capabilities of LLMs.
How It Works
The system works as follows: a user query enters the LLM agent, which parses the request and decides which domain-specific foundation models are relevant to answer it. The LLM then formats the query into the input specification for those models (e.g., SMILES strings for molecular models, sequence formats for protein models), invokes them, and receives back non-linguistic outputs. The agent then interprets these outputs in a semantic space the LLM can reason about, potentially calling additional models or performing multi-step reasoning chains. The core mechanism is a learned adapter layer or prompt engineering strategy that translates between the LLM's token space and the input/output specifications of specialized foundation models. This creates a heterogeneous compute graph where the LLM acts as a control plane and orchestrator, while domain models execute specialized inference in their native spaces.
Production Impact
For engineers building scientific AI systems, Eywa eliminates the costly choice between using text-only LLMs (which lose domain signal) or building custom multimodal models (high engineering cost). You can now compose existing foundation models (e.g., open-source protein folding tools, chemistry models, genomics models) under an LLM orchestrator, reducing the need to train monolithic models from scratch. In practice, this means faster time-to-market for scientific applications—you design an agent prompt and model routing logic rather than collecting massive multimodal datasets. The trade-off is increased inference latency (multiple model calls per query) and complexity in managing heterogeneous model versions, model rollouts, and debugging cross-model failures. A production pipeline would require robust orchestration layer, input validation to prevent malformed requests to specialized models, and careful API design for each domain model integration.
Limitations and When Not to Use This
The paper assumes each domain-specific foundation model has a well-defined, stable input/output interface, which is not always true in practice—scientific models often change signatures or improve with retraining. The approach requires the LLM to learn the 'language' of each domain model's outputs, which may fail for highly specialized or novel model types not seen during training, limiting generalization to new scientific tools. Scalability is unclear: with many domain models in the system, the LLM agent faces combinatorial explosion in deciding which models to invoke, and coordination overhead could dominate latency. The paper does not address data provenance or model trust—if a domain model produces incorrect results, the LLM may propagate errors downstream without flagging them. Finally, the work assumes access to pre-trained domain foundation models; fields without established foundation models cannot immediately benefit from this framework.
Research Context
This work builds on the broader trend of agentic LLMs (like ReAct, Chain-of-Thought prompting) that frame language models as reasoning engines that invoke tools, and extends this paradigm beyond single-modality tools (APIs, calculators) to multi-modality scientific tools. It addresses a documented limitation in LLM-agent systems: that language-only interfaces are lossy for scientific reasoning, a problem explored in recent work on scientific LLMs and vision-language models. Eywa contributes to the emerging research direction of modular foundation model ecosystems—instead of one monolithic model for all tasks, combining specialized models under a coordinator. The approach opens new research into better prompt/adapter design for cross-modal communication, formal methods for verifying agent behavior across heterogeneous systems, and benchmarks for evaluating multi-model orchestration in scientific domains.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::