Background: In this paper we present a detailed scheme for annotating medical web pages designed for health care consumers. The annotation is along two axes: first, by reliability (the extent to which the medical information on the page can be trusted), second, by the type of page (patient leaflet, commercial, link, medical article, testimonial, or support). Results: We analyze inter-rater agreement among three judges for each axis. Inter-rater agreement was moderate (0.77 accuracy, 0.62 F-measure, 0.49 Kappa) on the page reliability axis and good (0.81 accuracy, 0.72 F-measure, 0.73 Kappa) along the page type axis. Conclusions: We have shown promising results in this study that appropriate classes of pages can be developed and used by human annotators to annotate web pages with reasonable to good agreement. Availability: No.
CITATION STYLE
Martin, M. J. (2011). Reliability and type of consumer health documents on the World Wide Web: An annotation study. Journal of Biomedical Semantics, 2(3). https://doi.org/10.1186/2041-1480-2-S3-S5
Mendeley helps you to discover research relevant for your work.