Audit of data redaction practices in NICE technology appraisals from 1999 to 2019

Abstract

Objectives To assess the extent and type of data redaction in all active technology appraisals (TA) and highly specialised technology (HST) evaluations issued by the National Institute for Health and Care Excellence (NICE) from its conception of the institute to September 2019. To propose policy recommendations for transparency. Methods Structured audit to establish extent of data redaction—proportion of appraisals and specific data categories and assess redaction by: indication, appraisal process, manufacturer, type of data—price, adverse events (AEs), clinical (excluding AEs), incremental quality-adjusted life-years. Longitudinal analysis over 20 years. Results All TAs with available documentation and active recommendations (n=408) and HSTs (n=10) published from March 2000 to 11 September 2019 have been assessed for data redaction. Overall, 333 TAs (81.6%) have data redaction, 86 (25.8%) of them are heavily redacted. Clinical data (excluding AEs) are redacted in 268 (65.7%) appraisals, AE data in 128 (31.4%), price in 238 (58.3%). In total, 87% of oncology appraisals have redacted data vs 78% of non-oncology appraisals. 91% of single TAs have redacted data vs 59% of multiple TAs. 25% of final guidance documents (e.g. Final Appraisal Determination - FAD) do not report one or more instance of clinical data. Data redaction increased substantially over time, and is currently at its highest level with 100% of TAs having at least some data redaction in 2019/2020, 96% of appraisals in 2018/2019% and 94% of appraisals in 2017/2018. All 10 HST evaluations have redacted data, with 4 of them being heavily redacted. Conclusions Documents supporting NICE TA and HST recommendations are significantly redacted, thereby concealing clinical and economic data of importance to patients, clinicians and researchers. Documents remain redacted on the NICE website for years. Policy change is required to ensure transparency of data underpinning NICE’s decisions.