Applying the semantic graph approach to automatic essay scoring

Abstract

Frequent quizzes are used to motivate students to study throughout the academic term instead of waiting until just before the examination. Teachers have been relying on the use of multiple-choice questions to reduce their grading effort. We have developed a tool that grades answers of essay questions automatically. The tool should be a welcome addition to a teacher’s arsenal for quiz preparation. The teacher will provide the model answer of an essay question. Incorporating some heuristics into the Stanford Parser, our tool recognizes the parts of speech of each word and creates a parse tree from each sentence. It builds a semantic graph from the sentences in the model answer. The graph uses nodes to represent words and phrases. A directed arc connects two nodes to represent a relation. Currently, four types of arcs are used: attribute, possession, classification and action. In the same way, our tool determines the semantic graph of the student answer. Our tool compares the graph of the model answer with the graph of the student answer. It calculates a score to reflect their similarity. The relative weights of nodes and arcs are adjustable. WordNet helps us to identify synonyms so that the two answers need not be using the same wording to be considered similar. Our tool has some limitations. First, our semantic graph cannot handle timing sequences, for example, “event A happens before event B”. Second, our graph cannot handle conditional knowledge like “if X, then Y”. In the future, we may be able to introduce new arc types to address these limitations. Our prototype is not yet ready to replace the grading performed by teachers in formal assessment. But it may be useful to allow students to check their understanding during their self-study.