GAViD: A Large-Scale Multimodal Dataset for Context-Aware Group Affect Recognition from Videos
| Authors | Deepak Kumar et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2604.16214 |
| Download | |
| Categories | cs.CV |
Abstract
Understanding affective dynamics in real-world social systems is fundamental to modeling and analyzing human-human interactions in complex environments. Group affect emerges from intertwined human-human interactions, contextual influences, and behavioral cues, making its quantitative modeling a challenging computational social systems problem. However, computational modeling of group affect in in-the-wild scenarios remains challenging due to limited large-scale annotated datasets and the inherent complexity of multimodal social interactions shaped by contextual and behavioral variability. The lack of comprehensive datasets annotated with multimodal and contextual information further limits advances in the field. To address this, we introduce the Group Affect from ViDeos (GAViD) dataset, comprising 5091 video clips with multimodal data (video, audio and context), annotated with ternary valence and discrete emotion labels and enriched with VideoGPT-generated contextual metadata and human-annotated action cues. We also present Context-Aware Group Affect Recognition Network (CAGNet) for multimodal context-aware group affect recognition. CAGNet achieves 63.20% test accuracy on GAViD, comparable to state-of-the-art performance. The dataset and code are available at github.com/deepakkumar-iitr/GAViD.
Engineering Breakdown
Plain English
This paper introduces GAViD (Group Affect from Videos), a large-scale annotated dataset for understanding how emotions and social dynamics emerge in group settings captured on video. The authors tackle a critical gap in computational social systems: existing datasets lack multimodal (audio, video, text) annotations combined with contextual information needed to model real-world group interactions. The core contribution is both a dataset resource and a framework for analyzing affective dynamics—how individual emotions and behaviors shape collective group mood—in unconstrained, real-world scenarios. This directly addresses the bottleneck that has limited progress in group affect modeling, which requires training data that captures the complexity of human interaction.
Core Technical Contribution
The primary novelty is the GAViD dataset itself, which represents the first comprehensive, large-scale resource combining multimodal video data with dense annotations of group-level affect and contextual metadata. Rather than proposing a new model architecture, the authors create the foundational dataset infrastructure that enables future model development by capturing the inherent complexity of group dynamics—behavioral cues, contextual influences, and inter-personal interactions—in a unified annotation scheme. The dataset design explicitly addresses the multimodal and contextual variability challenge that prior small-scale datasets could not represent. This is a data contribution that unblocks downstream research similar to how ImageNet or COCO catalyzed progress in their respective domains.
How It Works
The GAViD dataset collection process begins with in-the-wild video recordings of group interactions across diverse settings and social contexts. Each video is annotated with multiple modalities: visual information (facial expressions, body language, spatial positioning of individuals), audio cues (tone, speech patterns, sentiment), and temporal context (sequence of events, environmental factors). Annotators label group-level affect states (e.g., engagement, valence, arousal at the group level) and individual contributions to those states, capturing how single-person behavioral cues propagate to influence collective affect. The resulting dataset structure enables training models that learn the compositional mapping from individual multimodal signals and contextual factors to emergent group-level emotions, supporting both supervised learning and future self-supervised approaches that can discover interaction patterns.
Production Impact
For production systems modeling human interactions—such as team collaboration analytics, customer service quality monitoring, or social wellbeing applications—this dataset enables building deployed models that predict and understand group dynamics beyond simple individual sentiment aggregation. Engineering teams can now train multimodal models on a validated, large-scale corpus rather than collecting and annotating proprietary data, reducing time-to-deployment and enabling rapid iteration on architectures. The contextual annotations are particularly valuable for transfer learning: models trained on GAViD should generalize better to new group settings because the dataset captures situational variability that single-domain datasets miss. Trade-offs include the computational cost of processing multimodal data streams in real-time (requiring efficient audio-visual fusion), and the need for careful domain adaptation since group dynamics vary significantly across cultural, professional, and social contexts.
Limitations and When Not to Use This
The paper does not present novel computational methods—it is primarily a dataset contribution—so organizations looking for architectural innovations in affect modeling should supplement this work with state-of-the-art multimodal fusion techniques from the literature. The dataset's coverage of real-world scenarios, while broad, likely skews toward specific geographic, demographic, and cultural contexts that reflect where video data was collected, potentially limiting direct deployment in understudied populations or novel domains. Temporal modeling of affect dynamics over long sequences (how group mood evolves over hours or days) may be underdeveloped if annotations focus on shorter clips. Finally, the paper does not fully address the subjective nature of affect annotation—inter-annotator agreement rates and bias in labeling are crucial but may not be comprehensively documented, which would affect model reliability in production.
Research Context
This work addresses a well-recognized gap in affective computing and computational social science: prior datasets like AffWild, VoxCeleb, and MELD focus on individual emotion or dyadic interactions, but large-scale group affect resources were absent. The paper builds on decades of social psychology research on group dynamics (e.g., emotional contagion, collective behavior) and applies the dataset infrastructure paradigm successful in vision (ImageNet, COCO) and NLP (MNIST, GLUE) to a new domain. It opens a research direction in multimodal group understanding that could spawn work on fairness in group affect detection, cultural variations in collective emotion, and temporal models of social influence. The dataset likely becomes a benchmark for evaluating how well models capture emergent phenomena—properties that arise from interactions and cannot be predicted from individuals alone.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::