Measurement Reliability and Its Evaluation ─ Especially on Intraclass Correlation Coefficients and Kappa Coefficients

Abstract

In recent years, the use of medical information databases has increased. When conducting research using medical information databases, it is necessary to define outcomes, exposures, and confounding factors that align with the research objectives from the information in the database. At this time, validation is required to evaluate the extent to which true cases can be identified. The presence or absence of disease is determined by chart review by multiple raters, and inter-rater reliability is evaluated. As the use of medical information databases increases, opportunities to conduct such reliability studies will increase. Therefore, in this paper, we will review measurement reliability and how to estimate the intraclass correlation coefficient and kappa coefficient, which are used as reliability indicators, while also referring to recent research.