Estimating the Consistency and Accuracy of Classifications Based on Test Scores

  • Livingston S
  • Lewis C
  • 18

    Readers

    Mendeley users who have this article in their library.
  • 58

    Citations

    Citations of this article.

Abstract




This article presents a method for estimating the accuracy and consistency of classifications based on test scores. The scores can be produced by any scoring method, including a weighted composite. The estimates use data from a single form. The reliability of the score is used to estimate effective test length in terms of discrete items. The true-score distribution is estimated by fitting a 4-parameter beta model. The conditional distribution of scores on an alternate form, given the true score, is estimated from a binomial distribution based on the estimated effective test length. Agreement between classifications on alternate forms is estimated by assuming conditional independence, given the true score. Evaluation of the method showed estimates to be within 1 percentage point of the actual values in most cases. Estimates of decision accuracy and decision consistency statistics were only slightly affected by changes in specified minimum and maximum possible scores.

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

  • Samuel A. Livingston

  • Charles Lewis

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free