Appstrument - A unified app instrumentation and automated playback framework for testing mobile applications

Abstract

Mobile Test Automation is gaining significant importance for an app-tester because it helps to alleviate the voluminous effort and time associated in thoroughly testing an application. Challenges like diversity in mobile hardware, multiple operating systems, ever-increasing application complexity and high volume of test cases etc. reiterate the importance of exploiting automation techniques for mobile application testing. In order to exhaustively capture user actions during the record-phase, faithfully reproduce those actions during playback-phase and also to capture the relevant metrics while playing back, instrumentation of the Application-under-test (AUT) becomes an imperative process. However, the type and level of instrumentation is different and is very specific to the category of testing which has to be automated. This paper presents Appstrument , a unified framework for instrumenting mobile applications to make them ready for functional, performance and accessibility testing. This framework allows instrumenting the application to get it ready for either a single category of testing or a combination of two or more of these categories, with multiple optional features for each category. In addition to this, given a test script, the framework also supports automated playback of instrumented applications. Appstrument has been deployed and tested against some popular applications from Google Play (Android apps) and some IBM in-house iOS applications. Results indicate that this framework is able to successfully instrument a sizeable number of applications and effectively playback user-defined test cases automatically to collect relevant metrics/results corresponding to each category of testing.