Lessons Learned from Asynchronous Online Assessment Formats in CS0 and CS3

Abstract

This paper provides an experience report for the asynchronous online assessments in two classes in Fall 2020: CS0 and CS3. The two courses shared many structural similarities; both were taught by the same instructor, with three exams delivered asynchronously using the same submission and question randomization software. They differed in proctoring rules, time allowances, student feedback, and lessons learned. In both classes, the first exam followed the same format, with students being given unlimited time over a 24-hour period to take the exam; later exams diverged due to different preferences among students in each class. CS0 historically had extremely low rates of academic dishonesty, so no formal proctoring was mandated. Thankfully we did not find evidence of academic impropriety at a higher rate than in-person semesters. Students universally loved the untimed format due to the flexibility that it provided, which led us to increasing the time window in which students could take the exam. Cheating was more prevalent within CS3 during in-person semesters, so it was not appropriate to have unproctored exams. Instead, we required students to submit a recording of themselves taking the exam. Even with these proctoring policies in place, students still found ways to cheat. Additionally, many students reported that the unlimited time increased their stress, leading us to restrict the time policies as the semester progressed. Our takeaway is that multi-day unproctored randomized exams are effective in a student population with high honor-code adherence, but finding the right policy for a high-stress major course remains elusive.