Statistical Methods: Reliability Assessment and Method Comparison

Abstract

서 론 임상 또는 의학 연구에서 위험요인을 선별하거나 질병을 진단하 거나 또는 환자의 예후를 추정하기 위해 측정을 하게 되며, 적절 한 측정 방법, 검사법의 선택은 임상적 결정을 내리는 과정과 연 구의 질을 보장하는 데 있어 필수적이다. 검사법은 주로 타당도 와 신뢰도의 측면에서 평가되곤 한다. 타당도(validity)는 검사법 이 측정하고자 하는 바를 제대로 반영하는 능력이나 실제 모수의 값을 정확하게 관찰하는 능력을 의미하며 정확도(accuracy)와 동 일한 의미로 사용된다[1-3]. 신뢰도(reliability)는 검사 시기, 실험 실, 평가자 등 측정 조건과 상관없이 검사결과가 얼마나 일관되게 나타나는지의 일관성(consistency), 또는 측정 오차(measurement error)가 없는 것을 말하는 것으로, 반복성(repeatability), 재현성 (reproducibility), 일치도(agreement, concordance) 등이 동일한 The reliability of clinical measurements is critical to medical research and clinical practice. Newly proposed methods are assessed in terms of their reliability, which includes their repeatability, intra-and interobserver reproducibility. In general, new methods that provide repeatable and reproducible results are compared with established methods used clinically. This paper describes common statistical methods for assessing reliability and agreement between methods, including the intraclass correlation coefficient, coefficient of variation, Bland-Altman plot, limits of agreement, percent agreement, and the kappa statistic. These methods are more appropriate for estimating reliability than hypothesis testing or simple correlation methods. However, some methods of reliability, especially unscaled ones, do not clearly define the acceptable level of error in real size and unit. The Bland-Altman plot is more useful for method comparison studies as it assesses the relationship between the differences and the magnitude of paired measurements, bias (as mean difference), and degree of agreement (as limits of agreement) between two methods or conditions (e.g., observers). Caution should be used when handling heteroscedasticity of difference between two measurements, employing the means of repeated measurements by method in methods comparison studies, and comparing reliability between different studies. Additionally, independence in the measuring processes, the combined use of different forms of estimating, clear descriptions of the calculations used to produce indices, and clinical acceptability should be emphasized when assessing reliability and method comparison studies.