Learning management systems: What more can we know?

Abstract

Learning management systems (LMS) are ubiquitous among colleges and universities worldwide, however they are thought of as a transactional warehouse rather than an opportunity to understand student learning outside of the classroom. Each LMS is used differently across campuses, and even across sections of the same courses taught by different instructors. For instance, one instructor might utilize a gradebook feature which allows students to view their assignment grades, while another instructor in the same course might use a different method to distribute grades for assignments. But is there a differential relationship between tool usage and student engagement or performance in the course across these sections? Our research addresses this issue and seeks to understand the nature of how LMS tools are used by students and how the use of those tools may shed insight on student learning or engagement. We ground our work theoretically using the Academic Plan Model to understand how freshman engineering students' use of LMS tools relate to their performance in the class. The Academic Plan Model details potential influences on curriculum design at the course, program, and institutional levels. As the Model suggests, faculty members may (or should) consider learners, instructional resources, and instructional processes when developing their curricular plans. Prior research within and outside engineering, however, has shown that faculty tend not to draw on available data when considering these components, if they even consider them at all. Our study presents an idea for bringing data into those considerations by focusing on the course-level activities of students within an LMS. We empirically describe an opportunity educators have to understand what can be learned from investigating LMS student data. Specifically, our data set consists of student LMS log files (approximately 15 million rows) for one engineering course (36 sections and 876 students) from Fall 2013. Results show clear patterns of student engagement with different LMS tools across final grades within first year engineering courses. Additionally, certain tool usage relates more strongly with course performance. By understanding how and when students use those tools in particular, faculty members may be able to create more data-informed course plans and provide empirically driven feedback to students on their levels of engagement in the class.