PMD4 Expert Elicitation to Populate Early Health Economic Models of Medical Diagnostic Devices in Development

Abstract

Purpose: During the development of new diagnostic and therapeutic devices, it is desirable to indicate the cost-effectiveness through modeling and to establish its potential clinical value to guide further developments. However, in these early stages of development, there are usually no or sparse clinical data available. In this study, expert elicitation was used as a method to estimate uncertain priors of the diagnostic performance of a new imaging device, i.e. Photoacoustic Mammography (PAM). We compared PAM as an alternative to Magnetic Resonance Imaging (MRI) as a second line diagnostic in the detection of breast cancer. Method: Expert elicitation was used as a method to formulate the knowledge and beliefs of experts regarding the future performance of PAM and to quantify this information into probability distributions. 18 experienced radiologists (specialized, in examining MR-images of breasts) were asked to estimate the importance of different tumor characteristics in the examination of images of breasts. Following this, the performance of visualizing these characteristics were estimated for both MRI and PAM. Using the mathematical approach to elicitation, the radiologists estimated the true positive rate (TPR) and true negative rate (TNR) based on existing MRI data (with a TPR of 263 out of 292, and a TNR of 214 out of 308) and specified the mode (the most likely value), the lower, and the upper boundaries (a 95% credible interval). An overall probability density function (PDF) was determined using the linear opinion pooling method in which weighting is applied to reflect the performance of individual experts. Result: The elicited judgments show that the most important characteristics in the discrimination between benign and malign tissue are mass margins (30.44%) and mass shape (28.6%). The oxygen saturation (2.49%) and mechanical properties (9.48%) were less important as there is limited information available about the added value of these characteristics. The performance of MRI on visualizing mass margins and mass shape was estimated to be higher than PAM, where PAM scored higher in the performance of displaying oxygen saturation and mechanical properties. An overall score of MRI (82.28) and PAM (54.03) indicates that MRI performs best in visualizing lesions of the breast. From the expert elicitation process an overall sensitivity was estimated ranging from 58.9% to 85.1%, with a mode of 75.6%. The specificity ranged from 52.2% to 77.6%, with a mode of 66.5%. Radiologists expressed difficulties making the estimations, as they felt there was insufficient data about the manner in which PAM visualizes different tumor types. Conclusion: The examination of tumor characteristics indicates that PAM is inferior over MRI. However, if oxygen saturation and mechanical properties are more important in the examination of images of breasts, this results in higher performance of PAM. Using expert elicitation in the absence of clinical data, prior distributions of the range of sensitivity and specificity can be obtained. Theoretically, this data can be fed into early health economic models. There were, however, difficulties expressed by experts in estimating the performance of PAM, given the limited existing evidence and clinical experience. The expression of uncertainty surrounding their beliefs should reflect the infancy of the diagnostic method, however further clinical trials should be commissioned to indicate whether these results are valid. Before that, the use of the elicited priors in health economic models requires careful consideration.