Sensitivity and specificity of artificial intelligence with Microsoft Azure in detecting pneumothorax in emergency department: A pilot study

Abstract

Background: Artificial intelligence is becoming an increasingly important tool in different medical fields. This article aims to evaluate the sensitivity and specificity of artificial intelligence trained with Microsoft Azure in detecting pneumothorax. Methods: A supervised learning artificial intelligence is trained with a collection of X-ray images of pneumothorax from National Institutes of Health chest X-ray dataset online. A subset of the image dataset focused on pneumothorax is used in training. Two artificial intelligence programs are trained with different numbers of training images. After the training, a collection of pneumothorax X-ray images from patient attending emergency department is retrieved through the Clinical Data Analysis & Reporting System. In total, 115 pneumothorax patients and 60 normal inpatients are recruited. The pneumothorax chest X-ray and the resolution chest X-ray of the above patient group and a collection of normal chest X-ray from inpatients without pneumothorax will be retrieved, and these three sets of images will then undergo testing by artificial intelligence programs to give a probability of being a pneumothorax X-ray. Results: The sensitivity of artificial intelligence-one is 33.04%, and the specificity is at least 61.74%. The sensitivity of artificial intelligence-two is 46.09%, and the specificity is at least 71.30%. The dramatic improvement of 46.09% in sensitivity and improvement of 15.48% in specificity by addition of around 1000 X-ray images is encouraging. The mean improvement of AI-two over AI-one is 19.7% increase in probability difference. Conclusions: We should not rely on artificial intelligence in diagnosing pneumothorax X-ray solely by our models and more training should be expected to explore its full function.