A Survey of Controlled Experiments in Software Engineering

Abstract

Today, most processes, methods, and tools in software engineering only rely on experience, lacking empirical evidence for a practical application. For example, an unanswered question for most tools is Under which circumstances should the tool be used and when not?. For this reason, research should aim to show which technology is useful for whom for which tasks and in which environment [15]. For nding answers to those questions, controlled experiments can be used. This paper describes controlled experiments in software engineering including necessary steps for performing them and related challenges.