ETFPOS-IDF: A Novel Term Weighting Scheme for Examination Question Classification Based on Bloom's Taxonomy

Abstract

Numerous earlier studies focused on the term weighting scheme to increase examination question classification accuracy based on Bloom's Taxonomy (BT). While determining the cognitive level of the examination question, all the terms present in the question are not equally significant. Verbs are the most important parts of speech while assigning weights to the terms. However, two types of verbs may be present in the questions: BT and supporting. BT verbs have a higher impact on determining the cognitive level of a question than supporting verbs. Nevertheless, the proposed schemes of past studies assigned equal weight to both types of verbs. Therefore, this study aims to introduce the term weighting scheme ETFPOS-IDF, which assigns BT a higher weight than supporting verbs. The BT verbs were identified based on their position in the questions. Three datasets and three classifiers: Support Vector Machine, Artificial Neural Network, and Random Forest, were used in this study. Two evaluation metrics: accuracy and F1 score, were used to evaluate the performance of the proposed model. The experiment results showed that the proposed ETFPOS-IDF outperformed all the schemes introduced by earlier studies in examination question classification and achieved 0.749 in accuracy and 0.746 in F1 score. The finding of this study demonstrates that distinguishing between different verb types is significant in reducing the misclassification of examination questions. This research contributed by introducing a novel term weighting scheme in classifying examination questions based on BT. Future work may involve identifying the optimal weight for both types of verbs, evaluating the proposed scheme with a larger dataset, and comparing the performance with deep learning.