HealthMotionNet: Multi-Modal Feature Aggregation With Temporal Attention for Personal Exercise Health Assessment

Abstract

Personal exercise health assessment demands accurate, interpretable, and adaptive modeling of complex physiological dynamics. To address the limitations of conventional approaches, we propose HealthMotionNet, a robust framework that integrates multi-modal feature aggregation with temporal attention mechanisms. Central to our method is the Personalized Multi-Modal Health Model (PPDN), which models latent health states across physiologically interpretable subspaces—including cardiovascular load, muscular fatigue, metabolic stress, and hydration state. These dimensions are dynamically captured using personalized stochastic differential equations, drawing from multi-modal sensor observations and enabling individualized yet generalizable health state inference. To handle intra- and inter-person variability, the model employs a hierarchical variational inference strategy, leveraging multi-hypotheses posterior distributions for uncertainty-aware predictions. Complementing the health modeling module is the Adaptive Health Assessment Strategy (AHAS), which translates inferred latent states into real-time health risk evaluations. AHAS introduces a multimodal encoder and graphical propagation layer to adapt assessments based on temporal context, external conditions, and confidence estimates. It defines evidence-adjusted risk bands and context-sensitive control functionals to issue actionable feedback and forecast future risk under uncertainty. The integration of predictive attention and context-aware adaptation ensures responsiveness to dynamic exercise environments and user-specific conditions. Through extensive experiments on benchmark datasets, HealthMotionNet demonstrates superior accuracy, interpretability, and robustness compared to state-of-the-art baselines. This unified system offers a principled pathway toward trustworthy and personalized health monitoring, advancing the field of intelligent exercise assessment and context-aware decision support in health technology. To support real-world deployment, the proposed framework incorporates federated learning to enable privacy-preserving training across distributed wearable devices, especially when handling sensitive physiological signals such as ECG. To preserve privacy and ensure scalability, our system is implemented under a federated learning framework where a global encoder is collaboratively trained across distributed clients.