Pre-Training Multi-Concept Question Embeddings for Knowledge Tracing

Abstract

With the rise of online education platforms, the vast number of accumulated data conceal patterns regarding the evolution of students’ knowledge states. Tracking a student’s knowledge state through their historical interaction records is known as knowledge tracing, a critical task in intelligent educational systems. Current research predominantly focuses on designing high-performance networks to enhance knowledge state tracking capabilities, often employing simplistic methods for question embedding, such as one-hot encoding or graph-based representations. This paper proposes a pre-training model for multi-concept question embedding (Pre-training Multi-concept Question Embedding, PMQE), aimed at providing a robust upstream tool for knowledge tracing and intelligent education fields. We first leverage the textual information in the dataset, representing concepts using their names or descriptive text embeddings. We then utilize the structural information from the question–concept graph, applying graph convolutional networks to derive question embeddings that integrate both textual semantics and structural information. Additionally, auxiliary information (such as question difficulty) is utilized for joint model optimization. The model outputs include question embeddings, question–concept weight matrices, and question difficulty, which can be used for knowledge tracing and other downstream tasks. Experiments conducted on two real-world datasets and multiple models validate the effectiveness of our approach.