Systematic reviews as a metaknowledge tool: caveats and a review of available options

Abstract

The development of information and communication technologies accompanied by the exponential growth of electronic scholarly publications and computer archives has led, on the one hand, to the development of techniques for the systematic analysis of published literature and, on the other hand, to a significant growth in the number of literature reviews conducted and published. A systematic literature review seeks to comprehensively identify and synthesize all studies conducted on a given topic. However, how does one ensure that the entire state of the art on a given topic has been covered? How does one collect and select a large number of publications on a given topic, and how does one analyze the knowledge extracted from these publications? In other words, how do we use systematic literature analysis to produce new knowledge from existing knowledge? To make the best use of the various tools available today for conducting systematic reviews, it is necessary to be aware of their limitations. In the paper, we provide illustrations, examples, and detailed information on the various alternatives for conducting systematic reviews, presenting a review of available sources and tools. We report an analysis of the articles published in the journal International Transactions in Operational Research to show the potential and limitations of systematic review versus domain expert-based selection. Finally, we show that a good systematic review relies on the intelligent combination of the latest tools available for conducting systematic research with the knowledge of domain experts.