The Adoption of a Framework to Support the Evaluation of Gamification Strategies in Software Engineering Education

Abstract

Context: gamification has been largely used to increase the engagement and motivation of students and professionals in their organizations, with a variety of models/frameworks for developing gamified approaches. Problem: the empirical data published so far are not sufficient to elucidate the phenomena resulting from the use of gamification, as there is no standardization in the specification of evaluation strategies, methods of analysis and reporting of results. Objective: therefore, the objective of this study is to present and discuss the use of a framework for the evaluation of gamification in the context of software engineering education and training. Method: for this, we executed a case study, in which the framework was used to support the design of an evaluation study for a gamification case in a software process improvement research group in a public university. Results: We report the main findings from observations and reports from the applicator of the case study, and 11 recommendations for the design of evaluation studies supported by the framework. Our main findings are: providing examples of usage of the framework improves its understanding, the framework helped the applicator in understanding that qualitative and quantitative data could be use in compliment to each other, and it helped streamlining the design of the evaluation study, considering the consistency between data to be collected, evaluation questions, and the goals of the evaluation study.