The significance of positive verification in unit test assessment

Abstract

This study investigates whether computer science students' unit tests can positively verify acceptable implementations. The first phase uses between-subject comparisons to reveal students' tendencies to write tests that yield inaccurate outcomes by either failing acceptable solutions or by passing implementations containing bugs. The second phase uses a novel all-function-pairs technique to compare a student's test performance, independently across multiple functions. The study reveals that students struggle with positive verification and doing so is associated with producing implementations with more bugs. Additionally, students with poor positive verification produce similar number of bugs as those with poor bug identification.