Predicting Item Characteristic Curve (ICC) Using a Softmax Classifier

Abstract

The objective of item difficulty modeling (IDM) is to predict the statistical parameters of an item (e.g., difficulty) based on features extracted directly from the item (e.g., number of words). This paper utilizes neural networks (NNs) to predict a discrete item characteristic curve (ICC). The presented approach exploits one-to-one mapping from monotonically non-decreasing discrete ICCs to probability mass functions (PMFs). An NN was trained using soft labels for each item (by mapping ICCs to PMFs), with a softmax output layer representing PMF and the Kullback-Leibler divergence representing a loss function. Results of a cross-validation of the NN on 1742 retired logical reasoning items from the Law School Admission Test are presented and discussed.