Multiclass classification of four types of UML diagrams from images using deep learning

Abstract

UML diagrams are a recognized standard modelling language for representing design of software systems. For academic research, large cases containing UML diagrams are needed. One of the challenges in collecting such datasets is automatically determining whether an image is a UML diagram or not and what type of UML diagram an image contains. In this study, we collected UML diagrams from open datasets and manually labeled them into four types of UML diagrams (i.e., class diagrams, activity diagrams, sequence diagrams, and use case diagrams) and non-UML images. We evaluated the performance of five popular neural network architectures using transfer learning on the dataset of 3231 images that contains 700 class diagrams, 454 activity diagrams, 651 use case diagrams, 706 sequence diagrams, and 720 non-UML images, respectively. We also proposed our neural network architecture for multiclass classification of UML diagrams. The experiment results show that our proposed neural network architecture achieved the best performance amongst the algorithms we evaluated with an accuracy of 98.65%, a precision of 96.76%, a recall of 96.48%, and an F1-score of 96.62%. Moreover, among the neural network architectures that we have evaluated, our proposed architecture has the least parameters (around 2.4 millions) and spends the least time per image (0.0135 seconds per image using GPU) for classifying UML diagrams.