From silver bullets to philosophers' stones: Who wants to be just an empiricist?

Abstract

For a long time scientists have been committed to describe and organize information acquired by observations from the field. To improve the comprehension and testability of the observed information, Bacon's works proposed to organize the way that the experiences should be structured and somehow formalized, starting with the experimental method idea. From that point in time, the ideas regarding experimentation have been explored and evolved into different scientific areas, including physics, agriculture, medicine, engineering and social sciences among others. It has not been different in Software Engineering. By applying the scientific method to organize their experimental studies, software engineers have intensively worked to understand the application and evolution of software processes and technologies. Acquiring knowledge through different categories of experimental studies has supported researchers and practitioners to build a Software Engineering body of knowledge. Families of studies start to be planed and shared among the research community, composing a common research agenda to enlarge such body of knowledge. Based on this, evidence based software engineering is becoming a reality. Nowadays, besides the experimental studies, the experimentation approach represents an important tool to allow the transfer of software technology to the industry and to improve software processes. Empirical Software Engineering relates to "software engineering based on observation or experience". However, the significance of the word empirical1 does not have enough power to capture the ideas regarding the scientific method (experimentation). By being just empirical, software engineers rely on their own experience or myths to observe and report the field. In general, there is no consensus regarding the experimental studies formalization, no common terminology or theory, and replication is not an easy task since a common research agenda can not be identified. In this case, empirical sounds like an ad-hoc approach, used to explore some particular research interest. It can represent one of the first steps (Aristotelian approach) towards the scientific method, but still far from it. Our Software engineering community needs to go one step further, looking for ways to better spread and internalize knowledge regarding the scientific method. Observing other more mature areas of science (i.e. physics, medicine, and social sciences) with well defined terminology, theories, studies protocols and common research agenda, empirical results has low value. It can also represent a risk for the software engineering scientific area. Software Engineering should not be interpreted as just empirical neither observational. It must evolve to be scientifically sounded and, of course, experimental ! © Springer-Verlag Berlin Heidelberg 2007.