GEM-AI: A Generative AI Driven Zero-Shot Method for Group Emotion Recognition

Abstract

In recent years, AI models have demonstrated a growing ability to interpret complex human behavior, including emotional states. While vision-language models (VLMs) have shown significant capabilities in this area, their application to group emotion recognition remains a challenging task. This challenge is compounded by the limitations of traditional evaluation metrics, which, due to their binary logic, fail to capture the nuanced semantic relationships between different emotional states. To address this evaluation gap, our work pioneers a systematic method for evaluating cohesive group emotion, moving beyond the simple aggregation of individual states. We achieve this within a zero-shot paradigm, establishing a lightweight baseline that eliminates the need for model fine-tuning. We present GEM-AI: a group-based Emotion recognition method, a novel group emotion detection framework based on semantic similarity. We apply this framework to a comparative study of several open-source VLMs, including LLaVA, MiniCPM, Deepseek-VL, and Qwen-VL, by reformulating precision, recall, and F1-score into “soft metrics” for a more semantically coherent assessment. Our results demonstrate that, when assessed with the GEM-AI framework, these models achieve high performance, reaching semantic accuracy of approximately 80%. This reveals a sophisticated comprehension of group emotion that is significantly underestimated by conventional metrics, which achieved only around 60% accuracy in our experiments.