Prioritizing JUnit Test Cases Without Coverage Information: An Optimization Heuristics Based Approach

Abstract

Regression testing is an expensive activity and Test Case Prioritization (TCP) acts as an improvement mechanism for it. TCP techniques for object oriented programs need attention and in our study, we explored prioritization of JUnit test cases. Ten benchmark Java programs with their several mutated versions were studied. As collecting coverage information is a costly effort, we bypassed these steps and used optimization heuristics for ordering JUnit test cases at test method level. Our approach formulated a novel fitness objective which depends on the number of modified lines executed per unit of execution time. As regression testing is performed after some modification is done on an existing program, maximizing the execution of number of modified lines is highly lucrative. The test case prioritization problem was replicated in context of 0/1 Knapsack problem and then it was solved using Genetic Algorithm (GA). Our exploration also included application of Simulated Annealing and Ant Colony Optimization method for determining the best execution ordering of test cases. We examined the usage of Multi-objective GA by building another new fitness metric which aims to maximize the number of inheritance edges covered by a test case. Results indicate the superiority of optimization heuristics over other existing approaches. It appeared that multi-objective GA yielded better result than single objective prioritization. Among the single objective techniques, ACO performed best. To the best of our knowledge, this is the first study which explored all the above mentioned optimization heuristics for ordering JUnit test cases with the newly coined fitness intents.