ATHENA: automatically tracking hands expertly with no annotations

Abstract

Studying naturalistic hand behaviors is challenging due to the limitations of conventional marker-based motion capture, which can be costly, time-consuming, and encumber participants. Although markerless pose estimation exists—an accurate, off-the-shelf solution validated for hand-object manipulation is needed. We present Automatically Tracking Hands Expertly with No Annotations (ATHENA), an open-source, Python-based toolbox for three-dimensional (3-D) markerless hand tracking. To validate ATHENA, we concurrently recorded hand kinematics using ATHENA and an industry-standard optoelectronic marker-based system (OptiTrack). Participants performed unimanual, bimanual, and naturalistic object manipulation and we compared common kinematic variables like grip aperture, wrist velocity, index metacarpophalangeal flexion, and bimanual span. Our results demonstrated high spatiotemporal agreement between ATHENA and OptiTrack. This was evidenced by extremely high matches (R2 > 0.90 across the majority of tasks) and low root mean square differences (<1 cm for grip aperture, <4 cm/s for wrist velocity, and <5–10 for index metacarpophalangeal flexion). ATHENA reliably preserved trial-to-trial variability in kinematics, offering identical scientific conclusions to marker-based approaches, but with significantly reduced financial and time costs and no participant encumbrance. In conclusion, ATHENA is an accurate, automated, and easy-to-use platform for 3-D markerless hand tracking that enables more ecologically valid motor control and learning studies of naturalistic hand behaviors, enhancing our understanding of human dexterity. NEW & NOTEWORTHY An accurate, easy-to-use Python-based toolbox is shared to perform automated three-dimensional (3-D) tracking of the hands. When validated against an industry standard marker-based system, the toolbox demonstrated high spatiotemporal agreement and preserved trial-to-trial variability for tasks ranging from simple reaching to complex object manipulation behaviors. The toolbox offers reduced financial and time costs and does not require the use of markers that may encumber participant movements, thereby facilitating ecologically valid motor control studies of the hand.