Evaluation of multifaceted programs

Abstract

This chapter discusses the development and use of an evaluation framework byNPSMedicineWise. It describes some of the evaluationmethods used including: surveys to determine changes in general practitioner knowledge and attitudes; the statistical methods to determine changes in prescribing of medicines as recorded in the Pharmaceutical Benefits Scheme (PBS); analyses for calculation of financial savings; and the use of an Australian longitudinal study with linked data sets to determine long-term patient outcomes. Why evaluate education programs? How will the evaluation findings be used? What if the findings are not positive and what impact will this have on funding? NPS MedicineWise took a clear position at its inception that it must be accountable for its work and that it must be a learning organisation. As the program is substantially funded by the Australian Government, it was important to have summative evaluations to demonstrate that program objectives were being met. At the same time, it was also necessary for the program staff and management to continually improve their understanding of what works and what does not work through process and impact evaluation. This intelligence meant that programs could be changed during implementation and that future programs would benefit from the lessons of the past.