Certain performance values arising from mammographic test set readings correlate well with clinical audit

Abstract

Introduction Test sets have been increasingly utilised to augment clinical audit in breast screening programmes; however, their relationship has never been satisfactorily understood. This study examined the relationship between mammographic test set performance and clinical audit data. Methods Clinical audit data over a 2-year period was generated for each of 20 radiologists. Sixty mammographic examinations, consisting of 40 normal and 20 cancer cases, formed the test set. Readers located any identifiable cancer, and levels of confidence were scored from 2 to 5, where a score of 3 and above is considered a recall rating. Jackknifing free response operating characteristic (JAFROC) figure-of-merit (FOM), location sensitivity and specificity were calculated for individual readers and then compared with clinical audit values using Spearman's rho. Results JAFROC FOM showed significant correlations to: recall rate at a first round of screening (r = 0.51; P = 0.02); rate of small invasive cancers per 10 000 reads (r = 0.5; P = 0.02); percentage of all cancers read that were not recalled (r = -0.51; P = 0.02); and sensitivity (r = 0.51; P = 0.02). Location sensitivity demonstrated significant correlations with: rate of small invasive cancers per 10 000 reads (r = 0.46; P = 0.04); rate of DCIS (ductal carcinoma in situ) per 10 000 reads (r = 0.44; P = 0.05); detection rate of all invasive cancers and DCIS per 10 000 reads (r = 0.54; P = 0.01); percentage of all cancers read that were not recalled (r = -0.57; P = 0.009); and sensitivity (r = 0.57; P = 0.009). No other significant relationships were noted. Conclusion Performance indicators from test set demonstrate significant correlations with specific aspects of clinical performance, although caution needs to be exercised when generalising test set specificity to the clinical situation.