New roles of Lagrange multiplier method in generalizability theory: Inference of estimating the optimal sample size for teaching ability evaluation of college teachers

Abstract

Background Generalizability theory is widely used in psychological and educational measurement. Budget and cost are the problems that cannot be neglected in the measurement. When there is a budget constraint, the generalizability theory needs to consider how to design a measurement program with relatively high reliability and feasibility, which requires the optimal sample size to be estimated by some means. Lagrange multiplier method is a commonly used method for estimating the optimal sample size under budget constraints in generalizability theory. Unfortunately, to date, many formulas of estimating the optimal sample size for some more complex generalizability designs such as those with four facets or more facets have not been derived using the Lagrange multiplier method. Purpose The purpose of this article is to provide a detailed step-by-step derivation of the formula of estimating the optimal sample size for three typical complex generalizability designs using the Lagrange multiplier method under budget constraints in generalizability theory, which can demonstrate the new roles of the Lagrange multiplier method. Method This article derived the optimal sample size for teaching ability evaluation of college teachers with budget constraints in three generalizability designs such as the (s:t)×i, (s:t)× (i:v) and (s:t) × (i:v) ×o and explored their practical applications. By estimating the optimal sample size, the optimal generalizability design, which is more applicable in practice, can be compared to obtain. Findings (1) Using the Lagrange multiplier method, the optimal sample size for students and items under budget constraints in different generalizability design can be derived. (2) For an example, based on teaching ability evaluation of college teachers in China, these designs of (s:t) ×i, (s:t) × (i:v) and (s:t) × (i:v) ×o were used to obtain the optimal sample size, which indicates the Lagrange multiplier method can been used in practice. (3) Under budget constraints, the (s:t) × (i:v) is the optimal generalizability design. The optimal sample size of students is 17 for each teacher and the optimal sample size of items is 4 for each dimension. Conclusion The optimal sample size can be derived carefully using the Lagrange multiplier method under budget constraints in generalizability theory. The Lagrange multiplier method with new roles is worth recommending.