Assessing short-term individual consistency using IRT-based statistics

Abstract

This article proposes a procedure, based on a global statistic, for assessing intra-individual consistency in a test-retest design with a short-term retest interval. The procedure is developed within the framework of parametric item response theory, and the statistic is a likelihood-based measure that can be considered as an extension of the well-known lz person-fit index. The rationale for using and interpreting the proposed statistic is discussed, and an adapted standardized residual at the item level is also proposed to obtain clues about the possible causes of the detected inconsistency. The procedure is illustrated with a real-data example and a parallel simulation in the personality domain.