A Quantitative Study of NLP Approaches to Question Difficulty Estimation

Abstract

Question Difficulty Estimation from Text (QDET) received an increased research interest in recent years, but most of previous work focused on single silos, without performing quantitative comparisons between different models or across datastes from different educational domains. To fill this gap, we quantitatively analyze several approaches proposed in previous research, and compare their performance on two publicly available datasets. Specifically, we consider reading comprehension Multiple Choice Questions (MCQs) and maths questions. We find that Transformer-based models are the best performing in both educational domains; models based on linguistic features perform well on reading comprehension questions, while frequency based features and word embeddings perform better in domain knowledge assessment.