Is there an optimum number needed to retrieve to justify inclusion of a database in a systematic review search?

Abstract

Objective: To determine whether calculation of a ‘Number Needed to Retrieve’ (NNTR) is possible and desirable as a means of evaluating the utility of a database for systematic review. Methods: To determine an overall NNTR, eight systematic reviews were tracked to determine how many abstracts were retrieved compared to the number of articles meeting the inclusion criteria. An NNTR was calculated for each database searched to measure the utility of including it in systematic review searches. Results: Across eight systematic reviews, 17 378 abstracts were reviewed. Of these, 122 met the inclusion criteria for their reviews resulting in an overall NNTR of 142. Individual reviews had an NNTR range of 28–310. Three databases delivered unique results (medline, cinahl and globalhealth). The majority of the included studies appeared in multiple databases. Only five articles were found in a single database. Conclusions: This research offers a proof of concept of ‘NNTR’. While the eight review NNTRs varied widely, all were consistent with the range initially reported by Booth. Included articles consistently appeared in multiple databases, suggesting that duplicate abstracts should be screened first as these are likely to include highly relevant, high-quality results.