A Survey of Empirical Investigations on SSBSE Papers

Abstract

We present a survey based on papers published in the first two editions of the Symposium on Search-Based Software Engineering (2009 and 2010). The survey addresses how empirical studies are being designed and used by researchers to support evidence on the effectiveness and efficiency of heuristic search techniques when applied to Software Engineering problems. The survey reuses the structure and research questions proposed by a systematic review published in the context of search-based software testing and covering research papers up to 2007. A list of validity threats for SBSE experiments is also proposed and the extent to which the selected research papers address these threats is evaluated. We have compared our results with those presented by the former systematic review and observed that the number of Search-based Software Engineering (SBSE) research papers supported by well-designed and well-reported empirical studies seems to be growing over the years. On the other hand, construct, internal, and external validity threats are still not properly addressed in the design of many SBSE experiments.