Application and Adjustment of “don’t care” Values in t-way Testing Techniques for Generating an Optimal Test Suite

Abstract

Exhaustive testing is a type of testing that test all possible combination of parameter values to make sure the product is free from any possible faults and errors. However, employing exhaustive testing would be impossible due to the factors that includes time, labor, cost, and resource constraints. Thus, combinatorial t-way testing was introduced to complement exhaustive testing problem. The t-way testing generates small test list considering that each test case can cover the greatest number of interaction tuples based on the t-way coverage. Many works were done in the field of combinatorial test generation to generate an optimized test suite based on a given t-way interaction strength, since t-way testing is exceedingly complicated (NP-hard). The t-way techniques make use of “don’t care” values when generating optimal test suite, but most of research adopt the values without a critical look into the consequences around them that lead to a higher number of test cases generated. As such, this paper presents the influence of “don’t care” values in the field of t-way testing, we also show that this line of research is threatening to lead the area away from optimization rigor. A case study is given as our practical example, we applied each t-way techniques up to two times to show the impact of “don’t care” values. Moreover, we suggest to the proper way of adopting “don’t care” values for t-way testing strategies. Finally, our suggestion can be applying when implementing t-way strategy consciously adopt the adjustment to reduce test suite size.