User-adaptive preparation of mathematical puzzles using item response theory and deep learning

Abstract

The growing use of computer-like tablets and PCs in educational settings is enabling more students to study online courses featuring computer-aided tests. Preparing these tests imposes a large burden on teachers who have to prepare a large number of questions because they cannot reuse the same questions many times as students can easily memorize their solutions and share them with other students, which degrades test reliability. Another burden is appropriately setting the level of question difficulty to ensure test discriminability. Using magic square puzzles as examples of mathematical questions, we developed a method for automatically preparing puzzles with appropriate levels of difficulty. We used crowdsourcing to collect answers to sample questions to evaluate their difficulty. Item response theory was used to evaluate the difficulty of the questions from crowdworkers’ answers. Deep learning was then used to build a model for predicting the difficulty of new questions.