Programming Question Generation by a Semantic Network: A Preliminary User Study with Experienced Instructors

Abstract

Questions are widely used in various instructional designs in education. Creating questions can be challenging and time-consuming. It requires not only the expertise of the learning content but also the experience of the question designs and the overall class performance. A considerable amount of research in the field of question generation (QG) has focused on computer models that automatically extract key information from a given context and transform them into meaningful questions. However, due to the complexity of programming knowledge, there are only few studies that have explored the potential of Programming QG (PQG) where natural languages and programming languages are often interwoven to constitute an assessment unit. To investigate further, this study experiments with a hybrid semantic network model for PQG based on open information extraction and abstract syntax tree. Our user study showed that experienced instructors had significantly positive feedback on the relevance and extensibility of the machine-generated questions.