The Delphi Method for Graduate Research

Abstract

The Delphi method is an attractive method for graduate students completing masters and PhD level research. It is a flexible research technique that has been successfully used in our program at the University of Calgary to explore new concepts within and outside of the information systems body of knowledge. The Delphi method is an iterative process to collect and distill the anony- mous judgments of experts using a series of data collection and analysis techniques interspersed with feedback. The Delphi method is well suited as a research instrument when there is incom- plete knowledge about a problem or phenomenon; however it is not a method for all types of IS research questions. The Delphi method works especially well when the goal is to improve our understanding of problems, opportunities, solutions, or to develop forecasts. In this paper, we pro- vide a brief background of the Classical Delphi followed by a presentation of how it has evolved into a flexible research method appropriate for a wide variety of IS research projects, such as de- termining the criteria for IS prototyping decisions, ranking technology management issues in new product development projects, and developing a descriptive framework of knowledge manipula- tion activities. To illustrate the methods flexibility, we summarize distinctive non-IS, IS, and graduate studies Delphi research projects. We end by discussing what we have learned from using the Delphi method in our own research regarding this method's design factors and how it may be applied to those conducting graduate studies research: i) methodological choices such as a quali- tative, quantitative or mixed methods approach; ii) initial question degree of focus whether it be broad or narrowly focused; iii) expertise criteria such as technical knowledge and experience, capacity and willingness to participate, sufficient time, and communication skills; vi) number of participants in the heterogeneous or homogeneous sample, v) number of Delphi rounds varying from one to 6, vi) mode of interaction such as through email, online surveys or groupware, vii) methodological rigor and a research audit trail, viii) results analysis, ix) further verification through triangulation or with another sample, and x) publishing of the results. We include an ex- tensive bibliography and an appendix with a wide-ranging list of dissertations that have used the Delphi method (including brief research description, number of rounds and sam- ple size). The Delphi method is a flexi- ble, effective and efficient research method that can be successful used by IS graduate students to answer research questions in information systems and to rigorously advance the IS body of knowledge.