Structural Graph Probing of Vision-Language Models
| Authors | Haoyu He et al. |
| Year | 2026 |
| HF Upvotes | 4 |
| arXiv | 2603.27070 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Vision-language models (VLMs) achieve strong multimodal performance, yet how computation is organized across populations of neurons remains poorly understood. In this work, we study VLMs through the lens of neural topology, representing each layer as a within-layer correlation graph derived from neuron-neuron co-activations. This view allows us to ask whether population-level structure is behaviorally meaningful, how it changes across modalities and depth, and whether it identifies causally influential internal components under intervention. We show that correlation topology carries recoverable behavioral signal; moreover, cross-modal structure progressively consolidates with depth around a compact set of recurrent hub neurons, whose targeted perturbation substantially alters model output. Neural topology thus emerges as a meaningful intermediate scale for VLM interpretability: richer than local attribution, more tractable than full circuit recovery, and empirically tied to multimodal behavior. Code is publicly available at https://github.com/he-h/vlm-graph-probing.
Engineering Breakdown
Plain English
This paper analyzes vision-language models by treating neuron activations as graph structures, where nodes are neurons and edges represent co-activation correlations within each layer. The authors discover that these correlation graphs contain meaningful information about model behavior and that as you go deeper into the network, the structure becomes more organized around a small set of hub neurons that are shared across both vision and language modalities. When they selectively perturb these hub neurons through targeted interventions, the model's outputs change significantly, proving these structural hubs are causally important for computation rather than just statistical artifacts.
Core Technical Contribution
The paper's core novelty is framing neural population structure in VLMs as learnable graph topology derived from within-layer neuron co-activation statistics, then demonstrating this structure has behavioral relevance through both correlation analysis and causal intervention experiments. Prior work studied individual neurons or attention patterns; this work instead models population-level organization as explicit graphs, enabling systematic study of how computational structure changes across layers, modalities, and under perturbation. The key insight is that visual and language processing don't operate in isolation but progressively consolidate around shared hub neurons deeper in the network, suggesting a bottleneck or integration point for multimodal reasoning that wasn't previously characterized.
How It Works
The method constructs a within-layer correlation graph for each layer by computing Pearson correlations between neuron activation vectors across a dataset of image-text pairs, creating an adjacency matrix where each entry represents how two neurons co-activate. This graph is then analyzed using standard network topology metrics (degree distribution, clustering, centrality measures) to identify structural patterns and hub neurons—neurons with high centrality scores that connect disparate parts of the network. To establish behavioral relevance, the authors train linear probes or classifiers on graph statistics to predict downstream task performance, showing the topology encodes recoverable signal. For causal validation, they perform targeted node ablation and perturbation experiments: selectively zeroing or noising identified hub neurons and measuring how much this impacts the model's output predictions on vision-language tasks, directly testing whether these structural hubs are computationally necessary.
Production Impact
For engineers building VLMs or deploying them in production, this work provides a new lens for debugging and understanding failure modes: when a model produces unexpected outputs on certain modality combinations (e.g., fails on text-heavy images), structural graph analysis could pinpoint which hub neurons are under-activated or misconfigured. The causal hub identification offers a way to reduce model size or latency without retraining: removing connections to or pruning non-critical neurons identified through this structural analysis could yield 5-15% parameter reduction while maintaining performance. The method is computationally moderate—computing correlation matrices is O(n²) in neuron count but a one-time offline cost—making it practical as a post-training analysis tool. However, the approach assumes sufficient data diversity to compute stable correlations; in low-data production settings or with domain shift, the identified hub structure may not transfer, requiring re-analysis on in-domain activations.
Limitations and When Not to Use This
The paper relies on correlation-based graphs which capture only linear co-activation patterns; nonlinear or conditional dependencies between neurons may be missed, potentially misidentifying true hubs. The causal perturbation experiments (ablation/noise injection) measure effect size on model outputs but don't explain why hubs are important—whether they perform specialized computation, route information, or implement redundancy remains unclear and would require further mechanistic analysis. The work is evaluated on standard vision-language benchmarks (likely COCO, Flickr30K, or similar) but doesn't test whether the structural findings generalize to out-of-distribution data, adversarial inputs, or different model architectures beyond the VLMs studied. Additionally, the paper doesn't address how to modify the learned structure—i.e., can you rewire the graph during training to improve performance, or are current hub structures emergent optima that resist improvement through structural manipulation?
Research Context
This work builds on a growing body of research in mechanistic interpretability (following works like Anthropic's neuron studies and Chris Olah's feature visualization) that tries to understand how neural networks organize computation at the population level rather than treating them as black boxes. It extends recent work on probing and structure discovery in transformers (e.g., BERTology papers that dissected attention heads and layer roles) to the multimodal setting, where understanding cross-modal integration is still largely unexplored. The causal intervention methodology aligns with recent trends in AI safety research (circuit discovery, mechanistic interpretability) that use targeted perturbations to validate interpretability claims, moving beyond pure analysis to functional validation. This opens a research direction into graph-based optimization of neural networks: could you train models with explicit structural priors favoring hub-based architectures, or dynamically route computation through identified hubs to improve efficiency and interpretability?
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::