Applying data mining methods to generate formative feedback in team projects

Abstract

It is a challenging task to evaluate the performance of individual team members of a project, more so when there is unequal contributions by the team members. The project supervisor often provide formative feedback through assessment of weekly work logs against the agreed project plan. Automating this evaluation mechanism has the obvious advantage of providing timely feedback in an efficient manner otherwise not possible due to the workload on supervisors especially in the academic settings. We explore the design space of an automated formative assessment solution based on text data mining. We evaluate the performance of various machine learning techniques (both classification and regression versions), by tweaking their control parameters, based on the achieved accuracy of prediction. We showed that the KNN regression models, despite their simplicity, produce the most accurate result across different similarity methods. We verify these research findings by employing various data visualisation techniques such as learning curves, residual plots and goodness of fit.