Empirical evaluation on FBD model-based test coverage criteria using mutation analysis

Abstract

Function Block Diagram (FBD), one of the PLC programming languages, is a graphical modeling language which has been increasingly used to implement safety-critical software such as nuclear reactor protection software. With increased importance of structural testing for FBD models, FBD model-based test coverage criteria have been introduced. In this paper, we empirically evaluate the fault detection effectiveness of the FBD coverage criteria using mutation analysis. We produce 1800 test suites satisfying the FBD criteria and generate more than 600 mutants automatically for the target industrial FBD models. Then we evaluate mutant detection of the test suites to assess the fault detection effectiveness of the coverage criteria. Based on the experimental results, we analyze strengths and weaknesses of the FBD coverage criteria, and suggest possible improvements for the test coverage criteria. © 2012 Springer-Verlag.