Life sciences-inspired test case similarity measures for search-based, FSM-based software testing

Abstract

Researchers and practitioners alike have the intuition that test cases diversity is positively correlated to fault detection. Empirical results already show that some measurement of diversity within a pre-existing state-based test suite (i.e., a test suite not necessarily created to have diverse tests in the first place) indeed relates to fault detection. In this paper we show how our procedure, based on a genetic algorithm, to construct an entire (all-transition) adequate test suite with as diverse tests as possible fares in terms of fault detection. We experimentally compare on a case study nine different ways of computing test suite diversity, including measures already used by others in software testing as well as measures inspired by the notion of diversity in the life sciences. Although our results confirm a positive correlation between diversity and fault detection, we believe our results raise more questions than they answer about the notion and measurement of test suite diversity, which leads us to argue that more work needs to be dedicated to this topic.