An Optimized LSTM Model for Diagnosis Prediction of Lower Respiratory Tract Infections Using a Minimalistic Data Set

Abstract

Data scarcity presents a significant challenge in developing effective data-driven diagnostic systems, particularly in healthcare settings where comprehensive feature capture is often limited. In medical contexts, vital features may be absent or unmeasurable due to constraints in laboratory capabilities and limited data collection infrastructure. This study addresses these limitations by deriving a minimalistic feature set to predict lower respiratory tract infections (LRTI’s), including diseases like pneumonia, bronchitis, and tuberculosis, using essential but widely available features. We employ age, gender, and initial diagnosis at admission—key indicators extracted from the MIMIC-III dataset—to train a Long Short-Term Memory (LSTM) model, demonstrating that minimal but relevant features can still yield accurate predictive outcomes. Compared to conventional models like Bi-LSTM, Random Forest, and XGBoost, our LSTM approach achieves superior training and validation performance, with an F1 score of 90% even without extensive optimization. The study validates our model across both LRTI and non-LRTI cases, underscoring its potential as an efficient, reliable diagnostic tool for resource-limited healthcare environments.