Why android app testing falls short: empirical insights from open-source projects and a practitioner survey

Abstract

Android dominates the mobile operating system market, yet ensuring the quality and reliability of Android applications remains a persistent challenge. The diversity of devices, screen sizes, and OS versions complicates testing, leading to fragmented adoption of best practices. Despite advancements in automated testing, there is Limited empirical evidence on how developers test Android applications and the extent to which existing tools and frameworks are utilized effectively. In this paper, we aim to investigate the current state of Android app testing, identifying key challenges, Limitations, and best practices. Specifically, we assess the adoption of automated testing, test coverage levels, and the impact of testing practices on software quality. We conduct an experimental study on 2965 open-source Android apps, examining the quantity and coverage of the tests used for open-source Android app development. We further conduct a survey to gather more insights in testing practices from Android app developers and testers. The results reveal a limited adoption of testing among Android app developers, a restricted range of testing tools and frameworks being used, and low code and API coverage in testing. This investigation shows that current Android app testing practices are lacking the use of automated testing tools and embarks on a need for more awareness and adoption of state-of-the-art testing tools and techniques.