Identification of toxic herbs using deep learning with focus on the Sinomenium Acutum, Aristolochiae Manshuriensis Caulis, Akebiae Caulis

Abstract

Toxic herbs are similar in appearance to those known to be safe, which can lead to medical accidents caused by identification errors. We aimed to study the deep learning models that can be used to distinguish the herb Aristolochiae Manshuriensis Caulis (AMC), which contains carcinogenic and nephrotoxic ingredients from Akebiae Caulis (AC) and Sinomenium acutum (SA). Five hundred images of each herb without backgrounds, captured with smartphones, and 100 images from the Internet were used as learning materials. The study employed the deep-learning models VGGNet16, ResNet50, and MobileNet for the identification. Two additional techniques were tried to enhance the accuracy of the models. One was extracting the edges from the images of the herbs using canny edge detection (CED) and the other was applying transfer learning (TL) to each model. In addition, the sensitivity and specificity of AMC, AC, and SA identification were assessed by experts with a Ph.D. degree in herbology, undergraduates and clinicians of oriental medicine, and the ability was compared with those of MobileNet-TL's. The identification accuracies of VGGNet16, ResNet50, and MobileNet were 93.9%, 92.2%, and 95.6%, respectively. After adopting the CED technique, the accuracy was 95.0% for VGGNet16, 63.9% for ResNet50, and 80.0% for MobileNet. After using TL without the CED technique, the accuracy was 97.8% for VGGNet16-TL, 98.9% for ResNet50-TL, and 99.4% for MobileNet-TL. Finally, MobileNet-TL showed the highest accuracy among three models. MobileNet-TL had higher identification accuracy than experts with a Ph.D. degree in herbology in Korea. The result identifying AMC, AC, and SA in MobileNet-TL has demonstrated a great capability to distinguish those three herbs beyond human identification accuracy. This study indicates that the deep-learning model can be used for herb identification.