Results for a multi-center investigation of the effects of network latency on pedagogic efficacy

Abstract

Interactive web-based learning tools, such as engineering simulations, are becoming increasingly common. Universities find them cost-effective, and students find them convenient. Professors find web-based simulations effective to intuitively convey the complex cause-and-effect relationships that are central in engineering education. For example, moving a slider can be used to interactively see how changing a resistor's value changes current flow through a current divider. There are many studies investigating the effectiveness of interactive web-based learning materials, yet, little systematic investigation of the pedagogic impact of network delay. This paper, therefore, seeks to quantify the relationship that network latency, or delay, has upon student enjoyment and student comprehension. An interactive software application was designed purportedly to teach Fourier Analysis concepts, but embeds a secret delay between the time a student moves one of the interactive controls and the time that the screen updates. Different versions of the application were designed, each identical except for the delay. Students were randomly assigned application versions, ensuring double-blind test conditions. Students used the application while completing a short guided lesson that used the Socratic Method to intuitively teach Fourier Analysis. After completing the tutorial questions, which provide an objective assessment of student comprehension, students self-rated their comprehension and enjoyment, and recorded their program version number which encoded the delay. The data was least-squares fit to several different functions with varying degrees of freedom and residuals were computed. Data involving 281 students from four universities and one high school using eight equally-spaced delays from 0 to 420 ms was analyzed. A two-part piecewise linear function was found to have both a low number of degrees of freedom and low sum of residuals that suggest a "knee" in pedagogic efficiency exists. One knee at a 300ms delay describes self-rated comprehension and self-rated enjoyment tolerance to delays. A second knee exists at 60ms ± 30ms and describes objective comprehension. The difference in knee location suggests that our learning is maximally effective for cause-and-effect relationships when delay is minimized, but that our psychological tolerance for delay is much higher. This conflict between competence perception and objective reality impacts university information technology infrastructure and pedagogical software design. This is especially the case for the emerging field of long-distance web education. These studies expose flaws in perception-based assessment of these areas. Continued studies are planned to assess category-specific differences such as age, gender, and major.