Understanding Hypotheses Engineering in Software Startups through a Gray Literature Review

Abstract

Context: The higher availability of software usage data and the influence of the Lean Startup led to the rise of experimentation in software engineering, a new approach for development based on experiments to understand the user needs. In the models proposed to guide this approach, the first step is generally to identify, prioritize, and specify the hypotheses that will be tested through experimentation. However, although practitioners have proposed several techniques to handle hypotheses, the scientific literature is still scarce. Objective: The goal of this study is to understand what activities, as proposed in industry, are entailed to handle hypotheses, facilitating the comparison, creation, and evaluation of relevant techniques. Methods: We performed a gray literature review (GLR) on the practices proposed by practitioners to handle hypotheses in the context of software startups. We analyzed the identified documents using thematic synthesis. Results: The analysis revealed that techniques proposed for software startups in practice compress five different activities: elicitation, prioritization, specification, analysis, and management. It also showed that practitioners often classify hypotheses in types and which qualities they aim for these statements. Conclusion: Our results represent the first description for hypotheses engineering grounded in practice data. This mapping of the state-of-practice indicates how research could go forward in investigating hypotheses for experimentation in the context of software startups. For practitioners, they represent a catalog of available practices to be used in this context.