Automatic generation of GUI test inputs using user configurations

Abstract

GUI testing validates the functionality of a software-intensive system by exercising its GUI. Although much research on automatic generation of GUI test inputs has been conducted to reduce the cost of GUI testing, the current GUI test input generation techniques can miss testing the behavior of the system which is dependent on the user configuration, which may leave undetected the defects that appear only under a certain user configuration. In order to completely test the behavior of a system for all possible user configurations, this paper proposes a method that automatically generate GUI test inputs under all possible user configurations. Since testing all possible user configurations is infeasible for nontrivial systems, the method is designed such that the user can sample user configurations. Thus, the proposed method generates GUI test inputs for the behavior of the system dependent on user configurations in addition to the test inputs generated by the existing technique that does not consider user configurations. We implement our method as an automated tool for the Android framework and evaluate it with on five open-source Android apps. The evaluation results show that our method can indeed achieve additional code coverage while preserving code coverage achieved by the existing technique.