Reliability of counting actinic keratoses before and after brief consensus discussion. The VA Topical Tretinoin Chemoprevention (VATTC) Trial

Abstract

Objective: To assess the reliability of counts of actinic keratoses (AKs) and the effect of a brief joint discussion of discrepancies on that reliability. Design and Intervention: Seven dermatologists independently counted AKs on the face and ears before and after a brief joint discussion of discrepancies. Setting and Patients: A volunteer sample of 9 patients from the ongoing VA (Department of Veterans Affairs) Topical Tretinoin Chemoprevention (VATTC) Trial. All participating individuals are veterans and have had 2 or more keratinocyte carcinomas (basal or squamous cell carcinoma) in the 5 years before enrollment in the study. Main Outcome Measure: Standard deviation of estimates of the Poisson regression parameter for the dermatologists. Results: Substantial variation was found among the dermatologists in their AK counts. The SD of the parameter estimates for the dermatologists decreased from 0.45 to 0.24 after the brief joint discussion, a 47% decrease (P=.076). The variation attributable to the dermatologists also decreased substantially (X26 decrease, 94 to 12). Conclusions: Actinic keratoses are common, and there is a continuous spectrum of lesions that ranges from sun-damaged skin to squamous cell carcinoma in situ. Clinical distinguishing features may be difficult to delineate precisely. Counts of AK are commonly performed, but appear to be unreliable, even when performed by experienced dermatologists. Joint discussion of discrepancies may enhance the reliability of these counts, although substantial variation remains. Research that relied on these counts must be reevaluated in light of the marked variation among expert observers. Future studies should consider measures to assess and enhance reliability.