Evaluation of the ToxRTool's ability to rate the reliability of toxicological data for human health hazard assessments

Abstract

Regulatory agencies often utilize results from peer reviewed publications for hazard assessments. A problem in doing so is the lack of well-accepted tools to objectively, efficiently and systematically assess the quality of published toxicological studies. Herein, we evaluated the publicly available software-based ToxRTool (Toxicological data Reliability assessment Tool) for use in human health hazard assessments. The ToxRTool was developed by the European Commission's Joint Research Center in 2009. It builds on Klimisch categories, a rating system established in 1997, by providing additional criteria and guidance for assessing the reliability of toxicological studies. It also transparently documents the study-selection process. Eight scientists used the ToxRTool to rate the same 20 journal articles on thyroid toxicants. Results were then compared using the Finn coefficient and "AC1" to determine inter-rater consistency. Ratings were most consistent for high-quality journal articles, but less consistent as study quality decreased. Primary reasons for inconsistencies were that some criteria were subjective and some were not clearly described. It was concluded, however, that the ToxRTool has potential and, with refinement, could provide a more objective approach for screening published toxicology studies for use in health risk evaluations, although the ToxRTool ratings are primarily based on study reporting quality.