Did it work? - Analysis of ways to measure the impact of an afterschool robotics outreach program

Abstract

All over the nation afterschool programs are implemented to assist children. This paper focuses on the challenge of evaluating the outcomes of an afterschool robotics program that is an outreach program of the Austin Children's Museum. Students use LEGO Mindstorms to explore robotics. The program works with third through fifth grade students at inner-city elementary schools in weekly afterschool sessions for eight weeks at each school. The goals of the program include: enhanced academic skills such as graphing and visual discrimination; increased interest and motivation for future experiences and positive attitudes towards mathematics, science and technology; and greater awareness of technology/team work processes such as brainstorming, planning teamwork and troubleshooting. This paper presents the effort to assess the impact the program has on the students in the area of interest. The goals of the program are outside the scope of traditional classroom goals and therefore require assessment different from standardized tests. The assessment is brief due to the short program (eight sessions), the age of the students (third through fifth grade), and the voluntary nature of the program. It was essential to minimize time assessment to maximize the children's experience. Due to these young students' written comprehension level, traditional assessment techniques were not appropriate. These challenges led to an assessment that includes a short pre and post written quiz of the students' comprehension of content areas, pre and post verbal interview of the students, and in-session records of students' abilities to demonstrate understanding of the content discussed. This paper discusses the success and shortcoming of the different ways of implementing the assessment. The analysis focuses on the ability of the assessment to measure a change in students' skills or attitudes. The analysis also discusses how each of the assessment techniques impacted the program and provides insight in an area that many programs find challenging. © American Society for Engineering Education, 2009.