Large language models in orthopedics: An exploratory research trend analysis and machine learning classification

Abstract

Background: Large Language Models (LLMs) are set to transform orthopedic practice with promising applications and a growing body of research. This exploratory study analyzed research trends in orthopedic LLMs and validated a machine learning classifier for categorizing publications into predefined domains. We hypothesized that LLM-related research would exhibit distinct thematic trends, and that a machine learning classifier would be able to accurately categorize research domains. Methods: A bibliometric analysis of 140 Scopus-indexed publications (2019–2024) was performed using keyword co-occurrence and thematic clustering. Articles were categorized into five areas: Patient Education, Research and Ethics, Surgeon Education, Clinical Support, and Diagnostics and Radiology Interpretation. Machine learning classifiers were trained on TF-IDF vectorized text and evaluated using precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC-ROC). Exploratory projections using linear regression assessed the volume and growth trends within the five research areas. Results: The exploratory analysis revealed a substantial increase in LLM publications increased significantly from 28 in 2023 to 108 articles in 2024. The support vector machine (SVM) model outperformed others, achieving 82 % accuracy (AUC-ROC: 0.97), with high precision for categorizing research in Clinical Assistance Tools and strong recall for Diagnosis and Radiology Interpretation. Subgroup analysis showed that Patient Education achieved balanced performance (precision: 88 %, recall: 78 %, F1-score: 82 %), but overlapping terminology caused misclassifications between research and education domains. Temporal analysis predicted continued growth in these research domains, with Patient Education (+26 %) and Research and Ethics (+57 %) leading the way through 2027. Conclusion: LLMs are exploring advancements in patient engagement, surgeon training, and orthopedic research, but challenges in reliability and ethics require careful implementation. Future work should focus on real-world validation, specialty-specific applications, and integrating multimodal AI systems. The SVM classifier demonstrated robust capabilities, providing a valuable tool for navigating the growing body of literature.