Empirical test design strategies using natural language processing

Abstract

With the rise in the role played by computers and their ubiquity, both developers and consumers share responsibility for improvements in the digital platform. Thus, there is a need to reduce the load on both ends with the help of automation. This proposed system, provides a simplistic interface, where users can submit issues with applications and developers are provided with test cases before writing off on the solutions to these issues. It aims to intelligently identify preexisting similar issues, if any, to reduce redundancy, identify system requirements unsatisfied by these issues as well as identifying the criticality of the issues. It also seeks to generate test cases for functional requirements. The system uses a ‘Deep Averaging Network (DAN)’ model developed by Google for sentence similarity to detect similar issues, providing an accuracy of upwards of 89% for an in-house prepared test set of more than 100 entries. It also uses sentence dependencies and a ‘Satisfiability Modulo Theories (SMT)’ solver, for identifying parts of speech and generates appropriate test cases.