“Enslaved to the trapped data”: A cognitive work analysis of medical systematic reviews

Abstract

Systematic reviews are a comprehensive and parameterised form of literature review, found in most disciplines, that involve exhaustive analyses and rigorous interpretation of prior literature. Performing systematic reviews, however, can involve repetitive and laborious work in order to reach reliable standards. Strict guidelines and availability of published reviews make the task amenable to computerised assistance and automation using text mining, information extraction, and machine learning techniques. However, it is unclear which aspects of this Work Task are best suited for such support. This paper describes a three-month ethnographic study and Cognitive Work Analysis of the systematic reviews performed by a medical research group. Our findings show that the IR aspects of systematic reviews involve many tasks at two separate levels: 1) taxonomic organisation of documents and sub-document elements in relation to topic queries and domain-specific resources, and 2) extraction methods for structured summaries from the classified resources. This provides the basis for future work designing search tools with localised optimization and subtask automation to support specific phases of the process.