Deep-learning-based markerless tracking of distal anatomical landmarks in clinically recorded videos for assessing infant movement patterns associated with neurodevelopmental status

Abstract

Abnormal patterns in infants’ General Movements (GMs) are robust clinical indicators for the progression of neurodevelopmental disorders, including cerebral palsy. Availability of automated platforms for General Movements Assessments (GMA) could improve screening rate and allow identifying at-risk infants. While we have previously shown that deep-learning schemes can accurately track the longitudinal axes of infant limb movements (12 anatomical locations, 3 per limb), information about the distal limb segments’ rotational movements is important for making an accurate clinical assessment, but has not previously been captured. Here we show that training schemes that are highly successful at tracking trunk and proximal limb landmarks perform less well for the distal limb landmarks, and this problem is exacerbated when landmarks are more precisely defined in the training-set to capture rotational movements. Increasing the sample size to 26 videos using a mixture of laboratory and clinical data pre-selected for diversity of pose and video conditions in a ResNet-152 deep-net model was sufficient to permit accuracy of >85% for the distal markers, and overall accuracy of 98.28% (SD 2.29) across the 24 landmarks. This scheme is suitable to form the basis of an infant pose reconstruction algorithm that captures clinically relevant information for an automated GMA.