The selection of experts for (probabilistic) expert knowledge elicitation

Abstract

Several different EKE protocols are reviewed in this volume, each with their pros and cons, but any is only as good as the quality of the experts and their judgments. In this chapter a structured approach to the selection of experts for EKE is presented that is grounded in psychological research. In Part I various definitions of expertise are considered, and indicators and measures that can be used for the selection of experts are identified. Next, some ways of making judgements of uncertain quantities are discussed, as are factors influencing judgment quality. In Part II expert selection is considered within an overall policy-making process. Following the analysis of Part I, two new instruments are presented that can help guide the selection process: expert profiles provide structure to the initial search, while a questionnaire permits matching of experts to the profiles, and assessment of training needs. Issues of expert retention and documentation are also discussed. It is concluded that although the analysis offered in this chapter constitutes a starting point there are many questions still to be answered to maximize EKE’s contribution. A promising direction is research that focusses on the interaction between experts and the tasks they perform.