Hawthorne effect and mathematical software based learning: A meta-analysis study

Abstract

The use of mathematical software in learning has been widely applied. This meta-analysis study was conducted to examine the "Hawthorne effect", or the novelty of using mathematical software in learning. Related databases are sought for studies that qualify for examining the effects of using mathematical software compared to traditional teaching on students' mathematical abilities. Random-effect models with 95% confidence intervals (CIs) were estimated. 40 effect sizes were analyzed from 40 primary studies that have been published in journals, or national and international proceedings from 2010 to 2020, and a total of 3072 students were included in this meta-analysis. The Comprehensive Meta-Analysis (CMA) program is used to assist analysis. As a result of the study, the overall effect size was 0.886 with a standard error of 0.092. The results of the follow-up analysis showed that the duration of the treatment was related to the use of mathematical software. It was found that the use of mathematical software given in the span of 0 - 4 weeks (effect size = 1.193) was more effective if carried out in more than 4 weeks (effect size = 0.535). Thus the Hawthorne effect influences the effectiveness of the use of mathematical software on students' mathematical abilities. This fact is considered by education practitioners in using mathematical software in the future.