On the placebo effect in interactive SBSE: A preliminary study

Abstract

Search Based Software Engineering approaches have proven to be feasible and promising in tackling a number of software engineering problems. More recently, researchers have been considering the challenges and opportunities related to involving users’ expertise in the resolution process, among other reasons, to deal with the mistrust or misunderstanding of fully automated optimisation approaches. This paper presents a preliminary study concerned at assessing the users’ subjective perception when his/her preferences are considered in an Interactive SBSE approach. Regarding the evaluation, we conducted a placebo-controlled study with 12 software engineering practitioners by simulating a Next Release Problem scenario. The results indicate that most (68%) of the gain achieved by the interactive approach could be attributed to being the placebo effect, that is, refers strictly to the fact that the user felt part of the optimisation process. In addition, there was an important increased confidence in the results, even in the placebo group.