Determining the sample size in agreement studies

Abstract

© 2017, Pacini Editore S.p.A. All rights reserved. Objective: Before beginning method comparison studies in clinical researches, all the researchers share a common problem. That is: how to determine the sample size. The aim of this study is to identify the sample size calculation steps for the consistency statistics used in the identification of the agreement between the raters/methods; and to present practical tables belonging to the minimum sample numbers required, before researchers start clinical trials. Materials and Methods: In this study, the steps of sample size calculation have been given for cases where there is no information on neither the population nor the consistency among the raters. Tables have been formed for Cohen Kappa and Intraclass correlation coefficient. Besides, other steps of sample number calculation have been given by utilizing a common formulation used for all consistency statistics by Gwet and practical tables have been presented. Results: When the tables are studied, no matter what the importance level and the test power is, the sample number increases while inconsistency rate between the two raters increases up to 0.50; and the sample number shows a symmetrical decrease while inconsistency rate between the two raters shows an increase from 0.50 through 1. Moreover, as the consistency value between the raters rise, no matter what the test power and the importance level is, the sample size to be included in the study decreases in direct proportion. Conclusion: Before beginning a research study, with the exact determination of the minimum number of samples enough for the design of the study and the state of the final variable, besides proving reliability of the results of the study, sampling waste will also be prevented.