Facilitate Robust Early Screening of Cerebral Palsy via General Movements Assessment with Multi-Modality Co-Learning

Abstract

General movement assessment (GMA) is a non-invasive method used to evaluate neuromotor behavior in infants under six months of age and is considered a reliable tool for the early detection of cerebral palsy (CP). However, traditional GMA relies on the subjective judgment of multiple internationally certified physicians, making it time-consuming and limiting its accessibility for widespread use. Furthermore, artificial intelligence (AI) approaches may overcome these limitations but are usually based on motion skeletons and lack the ability to capture detailed body information. Here, we propose CoGMA (Collaborative General Movements Assessment), a novel multi-modality co-learning framework for GMA. By integrating multimodal large language model as auxiliary network during training, CoGMA incorporates four types of input data - skeleton data, clinical information, RGB video, and text descriptions - to enhance representation learning. During inference, however, CoGMA achieves efficient and accurate prediction using only skeleton data and clinical information. Experimental evaluations indicate that CoGMA demonstrates robust performance across both the writhing and fidgety movement stages, while also excelling in zero-shot evaluation of fidget movement, thereby mitigating the issue of limited training samples in this stage. This framework significantly enhances the GMA methodology and lays the groundwork for future advancements in early detection and research on infant neuromotor behavior. Additionally, to facilitate anonymized data sharing, we introduce InfantAnimator, a tool that generates non-identifiable videos while preserving essential motion features, thereby supporting broader research and collaboration. The code is available at GitHub: https://github.com/wwYinYin/CoGMA.