Programme evaluation training for health professionals in francophone Africa: Process, competence acquisition and use

Abstract

Background: While evaluation is, in theory, a component of training programmes in health planning, training needs in this area remain significant. Improving health systems necessarily calls for having more professionals who are skilled in evaluation. Thus, the Université de Ouagadougou (Burkina Faso) and the Université de Montréal (Canada) have partnered to establish, in Burkina Faso, a master's-degree programme in population and health with a course in programme evaluation. This article describes the four-week (150-hour) course taken by two cohorts (2005-2006/2006-2007) of health professionals from 11 francophone African countries. We discuss how the course came to be, its content, its teaching processes and the master's programme results for students. Methods: The conceptual framework was adapted from Kirkpatrick's (1996) four-level evaluation model: reaction, learning, behaviour, results. Reaction was evaluated based on a standardized questionnaire for all the master's courses and lessons. Learning and behaviour competences were assessed by means of a questionnaire (pretest/post-test, one year after) adapted from the work of Stevahn L, King JA, Ghere G, Minnema J: Establishing Essential Competencies for Program Evaluators. Am J Eval 2005, 26(1):43-59. Master's programme effects were tested by comparing the difference in mean scores between times (before, after, one year after) using pretest/post-test designs. Paired sample tests were used to compare mean scores. Results: The teaching is skills-based, interactive and participative. Students of the first cohort gave the evaluation course the highest score (4.4/5) for overall satisfaction among the 16 courses (3.4-4.4) in the master's programme. What they most appreciated was that the forms of evaluation were well adapted to the content and format of the learning activities. By the end of the master's programme, both cohorts of students considered that they had greatly improved their mastery of the 60 competences (p < 0.001). This level was maintained one year after completing the master's degree, except for reflective practice (p < 0.05). Those who had carried out an evaluation in the intervening 12 months reported a negative gap between their declared mastery and their actual application. However, this is only statistically significant for reflective practice (p < 0.05). Conclusion: This study shows the importance of integrating summative evaluation into the learning process. Skills-based teaching is much appreciated and well-adapted. Creating a master's programme in population and health in Africa and providing training in evaluation to high-level health professionals from many countries augurs well for scaling up the practice of evaluation in African health systems. © 2009 Ridde et al; licensee BioMed Central Ltd.