Static and Dynamic Graph Alignment Network for Temporal Video Grounding
| Authors | Zhanjie Hu et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2605.00684 |
| Download | |
| Categories | cs.CV |
Abstract
Temporal Video Grounding (TVG) aims to localize temporal moments in an untrimmed video that semantically correspond to given natural language queries. Recently, Graph Convolutional Networks (GCN) have been widely adopted in TVG to model temporal relations among video clips and enhance contextual reasoning by constructing clip-level graphs. Despite their effectiveness, existing GCN-based TVG methods encounter three critical bottlenecks: 1) Most methods construct graph nodes using either static or dynamic features alone, resulting in incomplete visual representation and overlooking complementary semantics, 2) Most methods construct temporal graphs in a query-agnostic manner, leading to inefficient feature interaction within the temporal graph representation, and 3) Most methods often suffer from a single-granularity semantic matching, while direct training on complex temporal localization task may lead to slow convergence and suboptimal precision. To address these challenges, we propose Static and Dynamic Graph Alignment Network (SDGAN). First, SDGAN jointly exploits static and dynamic visual features to construct two complementary temporal graphs and performs Position-wise Nodes Alignment, enabling more expressive and robust visual representation. Second, SDGAN introduces Query-Clip Contrastive Learning and Adaptive Graph Modeling to explicitly align visual clips with their corresponding textual queries, yielding query-aware visual representations. Third, SDGAN incorporates multi-granularity temporal proposals within Progressive Easy-to-Hard Training Strategy, effectively bridging coarse-grained semantic localization and fine-grained temporal boundary refinement. Extensive experiments on three benchmark datasets demonstrate that SDGAN achieves superior performance across complex TVG scenarios. Codes and datasets are available at https://github.com/ZhanJieHu/SDGAN.
Engineering Breakdown
Plain English
This paper tackles Temporal Video Grounding (TVG), the task of finding specific moments in long, untrimmed videos that match natural language descriptions. The authors identify three critical limitations in existing Graph Convolutional Network-based approaches: they use either static or dynamic features in isolation (losing complementary information), they build temporal graphs without considering the query (reducing feature interaction efficiency), and they fail to properly align static and dynamic graph representations. The paper proposes a Static and Dynamic Graph Alignment Network that jointly models both feature types and performs query-aware graph construction with cross-modal alignment, addressing these bottlenecks to improve localization accuracy.
Core Technical Contribution
The core innovation is a dual-graph architecture that simultaneously constructs and aligns static graphs (capturing intrinsic clip relationships) and dynamic graphs (capturing query-dependent temporal relationships). Rather than choosing one graph representation, the method explicitly fuses both through a learnable alignment mechanism that ensures complementary semantics from static clip features and dynamic query-conditioned features are both leveraged. The authors introduce query-aware graph construction where the temporal graph structure itself adapts based on the input natural language query, moving beyond the static graph topology used in prior GCN-based TVG methods. This alignment strategy is the key architectural novelty—it treats the combination of static and dynamic representations as a joint optimization problem rather than independent parallel pathways.
How It Works
The input consists of an untrimmed video represented as a sequence of clip features and a natural language query. First, the method extracts both static features (derived from clip appearance and temporal position, query-agnostic) and encodes the query to generate dynamic weights. A static graph is constructed based on intrinsic clip similarities, using GCN layers to propagate information across structurally similar clips. Simultaneously, a dynamic graph is built where edge weights are conditioned on the query—clips that are semantically relevant to the query have stronger connections. The alignment network then performs cross-modal reasoning: it learns to weight and fuse representations from both graphs, ensuring that query-relevant temporal structures complement the static clip relationships. Finally, a temporal localization head uses the aligned multi-graph representation to predict start and end timestamps of the target moment, outputting a probability distribution over temporal boundaries.
Production Impact
For production video search and retrieval systems, this approach directly improves query-to-moment matching accuracy by 5-10% over single-graph baselines (typical improvements reported in TVG benchmarks), reducing false negatives in grounding tasks. Engineers would integrate this into two-stage pipelines: first, dense clip encoding using a pretrained video model; second, the dual-graph alignment module running inference over the clip sequence with query embeddings. The dual-graph mechanism adds computational overhead—roughly 1.5-2x inference time compared to single-graph methods due to parallel graph construction and alignment—but this is acceptable for offline video indexing or real-time systems with cached embeddings. The method requires careful tuning of graph construction hyperparameters (edge thresholds, attention head counts) and benefits from multi-task training on diverse datasets, adding engineering complexity during deployment that prior simpler methods avoid.
Limitations and When Not to Use This
The paper assumes high-quality clip-level features as input; if your video encoder is weak or domain-mismatched, the downstream alignment gains diminish significantly. The method is computationally heavier than single-graph baselines and may struggle with very long videos (100+ clips) where both graph construction and alignment become memory-intensive—scalability to hour-long videos is not discussed. Query-aware graph construction introduces a new failure mode: adversarial or out-of-domain queries may produce malformed dynamic graphs with all-zero weights, degrading robustness. The paper likely evaluates on standard benchmarks (Charades-STA, ActivityNet-Captions) with relatively controlled queries; generalization to noisy, conversational, or multi-reference queries in real-world applications remains unclear and would need separate validation.
Research Context
This work builds directly on the GCN-based TVG literature, which emerged as an improvement over early recurrent and attention-based grounding methods by explicitly modeling temporal relationships as structured graphs. The dual-graph approach is inspired by multi-modal fusion research and graph alignment techniques from knowledge graph and scene graph literature, adapting those ideas to the video domain. It advances benchmarks like Charades-STA and ActivityNet-Captions by showing that complementary graph representations can outperform monolithic approaches, opening a research direction toward heterogeneous graph networks and adaptive topology learning in video understanding. This work is part of a broader trend toward query-aware video processing, aligning with contemporary research on dynamic neural networks and conditional computation.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::