Simulation of Postmarket Fine-tuning of a Computer-aided Detection System for Bone Scintigrams and Its Performance analysis

Abstract

In this study, we performed simulations for bone scintigrams before and after a hot spot detection support system was fine-tuned using a postmarket dataset, and statistically identified the factors that affect the changes in performance. Datasets from five hospitals were used to train the premarket system, and the dataset from another hospital was added to fine-tune the system. We applied the premarket and postmarket fine-tuned systems to postmarket test data and computed the difference in the number of pixels of false positives and false negatives before and after fine-tuning. Structural equation modeling was used to analyze the relationship between the four possible factors and performance changes. The experimental results indicated that the image contrast and number of pixels of hot spots per image were the main factors affecting the performance. In addition, we identified the conditions for determining whether fine-tuning the system using postmarket datasets is appropriate. The experimental findings from this study will be useful for deriving an effective design scheme for continuous learning in artificial intelligence systems.