Unity application testing automation with appium and image recognition

Abstract

This work is dedicated to the problem of integrating simple functional tests (smoke tests) into the automated continuous integration pipeline. While functional testing is typically performed manually by the QA staff members, there is a large number of scenarios that can be automated, and readily available instruments, such as Appium and Calabash, designed for this task. Automated smoke testing is especially challenging for the applications with nonstandard GUI, such as games made with Unity. The only viable option in this case is to analyze screen content as a flat image and identify GUI elements with pattern matching algorithms. This approach is not novel, but its practical applicability and limitations are rarely discussed. We consider a case study of a mobile tennis game project, developed in Unity and covered with a suite of Appium-supported functional tests. We show how image matching capabilities of OpenCV library can be used in Appium tests to build a reliable automated QA pipeline.