International Journal of Osteoarchaeology Int. J. Osteoarchaeol. 12: 248���258 (2002) Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/oa.619 Analysis of Interobserver Scoring Patterns in Porotic Hyperostosis and Cribra Orbitalia KEITH P. JACOBIa* AND MARIE ELAINE DANFORTHb a Department of Anthropology and Alabama Museum of Natural History, University of Alabama, Tuscaloosa, USA b Department of Anthropology and Sociology, University of Southern Mississippi, USA ABSTRACT The role of scoring standards has become increasingly important during the last ten years because of issues such as reburial and comparability of data among researchers. The present study considered the efficacy of a proposed standard for porotic hyperostosis and cribra orbitalia, two of the most commonly evaluated pathologies in skeletal analysis. Twenty scorers with varying experience in bioarchaeology and five scorers with no experience in bioarchaeology evaluated 21 partial skulls for three characteristics: presence of pathology, appearance of porosities, and degree of healing. Participants showed good levels of agreement ( 80%) when a lesion was considered present, but most scorers never agreed that a specimen was free of pathology when, in fact, it was. Greater variation was seen in evaluation of porosity size among those cranial fragments with lesions. Determination of degree of healing showed even more diversity with 19 of 21 cases having all scoring options given. Level of experience did not appear to make a difference with respect to level of agreement. Virtually the same pattern of results was seen among the five individuals with no familiarity with porotic hyperostosis or cribra orbitalia but with some biological training. These findings suggest that further refinement, especially in the area of photographs and descriptions illustrating minimum and maximum representations of various scoring levels, are necessary to produce scoring standards for porotic hyperostosis and cribra orbitalia that are effective and reliable. Copyright ��� 2002 John Wiley & Sons, Ltd. Key words: porotic hyperostosis cribra orbitalia scoring standards palaeopathology interob- server error Introduction Standardization of scoring for skeletal pathologies has gained new importance during the last ten years with the implementation of policies such as reburial within the US and increased nationalism concerning collections in other areas of the world. Researchers have since evaluated patterns of scoring replicability for a number of conditions, including Harris lines (Grolleau-Raoux et al., 1997), linear enamel hypoplasias (Danforth et al., 1993), and osteoarthritis (Waldron & Rogers, * Correspondence to: Department of Anthropology and Alabama Museum of Natural History, University of Alabama, PO Box 870340, Tuscaloosa, Alabama 35487-0340, USA. 1991) as well as level of ability to discriminate among a number of pathological lesions (Miller et al., 1996). Porotic hyperostosis and cribra orbitalia are another commonly scored set of lesions in skeletal analysis, primarily because each has been directly or indirectly associated with anaemia (Stuart- Macadam, 1985 cf. Ortner & Putschar, 1981:259 cf. Aufderheide & Rodr��guez-Mart�� �� ��n, 1998:349), but the conditions have not yet been evaluated from the perspective of interobserver replication. It is the general tendency of researchers who docu- ment porotic hyperostosis, whether in a textbook, monograph, article or report, to exhibit a photo- graph of an individual with a more advanced case of the condition than is typically seen Copyright ��� 2002 John Wiley & Sons, Ltd. Received 25 August 1999 Revised 12 September 2000 Accepted 16 July 2001

Scoring Standards in Porotic Hyperostosis 249 (Angel, 1967 Aufderheide & Rodr�� ��guez-Mart�� ��n, 1998 Ortner & Putschar, 1981 Steinbock, 1976 Zimmerman & Kelley, 1982). Although strik- ing cases make for more dramatic photographs, mild expressions of porotic hyperostosis are what researchers overwhelmingly encounter in their analyses of skeletal populations. The omission of illustrations of slight-to-moderate cases, how- ever, potentially encourages underdiagnosis of the condition and a preconceived level of hyperse- verity in the minds of those scoring the lesions, especially among the less experienced. Lack of a scoring standard for porotic hyperostosis and cribra orbitalia also leads to complications when researchers begin to compare their own frequen- cies with those of others. It is understood that porotic hyperostosis and cribra orbitalia exhibit a range of expression. Buikstra & Ubelaker (1994) recently developed a scoring standard using photos that show gradients of porosity size and degree of healing in their manual, Standards for Data Collection from Human Skeletal Remains. It still needs to be determined, however, how useful the standards might prove to be, especially because individuals applying them may not have the same training and will have only a short written explanation as to how they should be implemented. The present study will test the working efficacy of these new standards in scoring replicability of porotic hyperostosis and cribra orbitalia among evaluators with a range of training in human osteology and then among those with no training. It is not intended to evaluate the criteria used to develop the standards themselves. The results may help us better understand what people physically recognize and code when they are asked to evaluate porotic hyperostosis and cribra orbitalia macroscopically. Previous coding criteria of porotic hyperostosis and cribra orbitalia Several coding criteria were developed for the analysis of porotic hyperostosis and cribra orbitalia in order to help identify the broad skele- tal manifestations of lesions, to evaluate their degree of severity, and to determine the pos- sible range of expression. The earliest attempt at partitioning the severity of lesions in cribra orbitalia can be found in the work of Welcker (1888) where, based on relative porosity size, severity is categorized as weak, stronger, and strongest. Nathan & Hass (1966a, 1966b) also offered three degrees of development of cribra orbitalia in primates: 1) porotic type with small pores, 2) cribrotic type with larger aggregated but still separated openings, and 3) trabecular type with openings that are coalesced and form trabec- ulae. Knip (1971:432) adds to these three types the important factor of healing where trabeculae are closed ������but the surface is crossed by sulci and depressions.������ Porotic hyperostosis also has been scored using various standards in the past. One of the earliest and perhaps one of the best descriptions of porotic hyperostosis was written by Hooton (1930) in his classic monograph on Pecos Pueblo. He described the lesions as occurring in symmetrical patches most commonly on the parietals, but also occa- sionally affecting adjacent bones as well. Hooton noted concomitant thickening of the dipl �� oe that ranged between 10 and 15 mm, and observed that it was possible to distinguish various degrees of healing and that active porotic hyperostosis was generally a childhood or adolescent con- dition. Since Hooton���s work, many researchers simply have evaluated the condition based on presence of porosities without overtly discussing minimum thresholds of expression (e.g., El Naj- jar et al., 1976). Others discuss subjective degrees of expression, such as slight/moderate/severe, but do not provide criteria for distinguishing among them (Angel, 1971). By the late 1970s, researchers also were incor- porating degree of healing into their scoring cri- teria, based largely on the work of Mensforth and colleagues (1978). Stuart-Macadam (1982), in her dissertation research, appears to have been one of the first to develop specific definitions for various degrees of severity of porotic hyperostosis. She later refined the scheme by including the variable of dipl �� oic thickening (Stuart-Macadam, 1985). Her definition included three severity levels: 1) light: with scattered fine foramina, 2) medium: large and small isolated foramina with some of the foramina coalesced to form trabeculae, and 3) severe: outgrowth in trabecular structure from the normal contour of the outer bone table. This definition was adapted by Buikstra & Ubelaker Copyright ��� 2002 John Wiley & Sons, Ltd. Int. J. Osteoarchaeol. 12: 248���258 (2002)