Evaluating integrated assessment tools for policy support

Abstract

Integrated Assessment Modelling tools are complex tools requiring specific evaluation methodologies. Based on the example of the SEAMLESS-Integrated framework, we show how the conceptual, technical and system evaluation steps of the different components (procedures, quantitative models, graphic user interfaces) were performed by a multidisciplinary team. To make the not-yet-available tool real, mock-up and test cases were mobilized throughout the development process in order to integrate final end-users in the evaluation process. The main lessons from the project are that the evaluation required: (i) the use of prototypes to advance properly in the design and testing (spiral methodology); (ii) the use of case studies to stick to the end-users requirements; (iii) a proper timing of development and delivery in order to keep on schedule and leave time to the evaluation process; (iv) a multidisciplinary team of evaluators as tools are of diverse types; and (v) that it is difficult to keep independence between testers, end-users and modellers in order to guaranty transparency in the development and evaluation process. © 2010 Springer Science+Business Media B.V.