Analysis of interobserver scoring patterns in porotic hyperostosis and cribra orbitalia

Abstract

The role of scoring standards has become increasingly important during the last ten years because of issues such as reburial and comparability of data among researchers. The present study considered the efficacy of a proposed standard for porotic hyperostosis and cribra orbitalia, two of the most commonly evaluated pathologies in skeletal analysis. Twenty scorers with varying experience in bioarchaeology and five scorers with no experience in bioarchaeology evaluated 21 partial skulls for three characteristics: presence of pathology, appearance of porosities, and degree of healing. Participants showed good levels of agreement (>80%) when a lesion was considered present, but most scorers never agreed that a specimen was free of pathology when, in fact, it was. Greater variation was seen in evaluation of porosity size among those cranial fragments with lesions. Determination of degree of healing showed even more diversity with 19 of 21 cases having all scoring options given. Level of experience did not appear to make a difference with respect to level of agreement. Virtually the same pattern of results was seen among the five individuals with no familiarity with porotic hyperostosis or cribra orbitalia but with some biological training. These findings suggest that further refinement, especially in the area of photographs and descriptions illustrating minimum and maximum representations of various scoring levels, are necessary to produce scoring standards for porotic hyperostosis and cribra orbitalia that are effective and reliable. Copyright © 2002 John Wiley & Sons, Ltd.