Automatically Grading Brazilian Student Essays

Abstract

Automated Essay Scoring (AES) is the NLP task of evaluating prose text, still scarcely explored in Portuguese. In this work, we present two AES strategies: the first with a deep neural network with two recurrent layers, and the second with a large number of handcrafted features. We apply our methods to evaluate essays from the ENEM exam with respect to five writing competencies. Overall, our feature-based system performs better in the first four, while the neural networks are better in the fifth one, which is also the hardest to grade accurately. In the aggregated score, our best model achieves a Quadratic Weighted Kappa of 0.752 and a Rooted Mean Squared Error of 100.0 when compared to human judgments, with scores ranging from 0 to 1000.