Paladin: Automated generation of reproducible test cases for android apps

Abstract

Automated-test-generation tools generate test cases to enable dynamic analysis of Android apps, such as functional testing. These tools build a GUI model to describe the app states during the app execution, and generate a script that performs actions on UI widgets to form a test case. However, when the test cases are re-executed, the apps under analysis often do not behave consistently. The major reasons for such limited reproducibility are due to (1) backend-service dependencies that cause non-determinism in app behaviors and (2) the severe fragmentation of Android platform (i.e., the alarming number of different Android OS versions in vendor-customized devices). To address these challenges, we design and implement Paladin, a novel system that generates reproducible test cases for Android apps. The key insight of Paladin is to provide a GUI model that leverages the structure of the GUI view tree to identify equivalent app states, since the structure can tolerate the changes on the UI contents for an app behavior performed in different test executions. Based on the model, Paladin can search the view tree to locate the desired UI widgets to trigger events and drive the app exploration to reach the desired app states, making the test cases reproducible. Evaluation results on real apps show that Paladin could reach a much higher reproduction ratio than the state-of-the-art tools when the generated test cases are re-executed across different device configurations. In addition, benefiting from the reproducible capability, Paladin is able to cover more app behaviors compared with the existing tools.