GUI Component Detection-Based Automated Software Crash Diagnosis

Abstract

This study presents an automated software crash-diagnosis technique using a state transition graph (STG) based on GUI-component detection. An STG is a graph representation of the state changes in an application that are caused by actions that are executed in the GUI, which avoids redundant test cases and generates bug-reproduction scenarios. The proposed technique configures the software application STG using computer vision and artificial intelligence technologies and performs automated GUI testing without human intervention. Four experiments were conducted to evaluate the performance of the proposed technique: a detection-performance analysis of the GUI-component detection model, code-coverage measurement, crash-detection-performance analysis, and crash-detection-performance analysis in a self-configured multi-crash environment. The GUI-component detection model obtained a macro F1-score of 0.843, even with a small training dataset for the deep-learning model in the detection-performance analysis. Furthermore, the proposed technique achieved better performance results than the baseline Monkey in terms of code coverage, crash detection, and multi-crash detection.