Development and validation of the NICE artificial intelligence (AI) medical device intervention search filters for MEDLINE and Embase (Ovid)

Abstract

Medical device interventions with an artificial intelligence (Al) component can potentially improve the effectiveness and efficiency of activities such as diagnostic screening through automation. However, inconsistent terminology for the interventions is used in research literature and it can be challenging to find evidence about them in literature searches reliably. Information specialists at the United Kingdom's National Institute for Health and Care Excellence (NICE) developed novel validated search filters to retrieve evidence about Al medical device interventions from MEDLINE and Embase (Ovid). Methods The filters were drafted using Al-related terminology selected from references identified in NICE Al medical device topics. Using relative recall methodology, gold standards containing references from systematic reviews about Al medical devices were generated for each database and then divided into Development and Validation Sets. The draft filters were finalised using their Development Sets and validated externally by calculating their recall against their Validation Sets. Target recall was >90 percent. Results Both filters achieved 96 percent recall against their Validation Sets. Conclusions The NICE Al medical device intervention search filters retrieve evidence about the interventions effectively and reliably. The filters can be used by information professionals, researchers, and clinicians to find evidence about Al medical device interventions in literature searches. The filters should be used as part of a 'multi-stranded' literature search strategy when details of brand names and/or medical device manufacturers are known.