Using augmented reality technology with serial learning framework to develop a serial social story situation board game system for children with autism to improve social situation understanding and social reciprocity skills

Abstract

This study adopts a Sequence Learning Strategy based on social stories and designs a sequential social story situation board game system using Augmented Reality (AR) technology (SL-ARS). In the system, the content of social stories is decomposed into social situation units with a Sequence Learning structure, different social situation judgment tasks are added into the stories, and autistic children are taught to think about the order of social events and situations that occur in their daily environment, correctly prioritize social situational states, and learn how to give appropriate social responses to others in a given situation. This study recruited three 7–9-year-old children with autism and observed the explicit behavior of autistic children during the operation of SL-ARS by micro-behaviors for video coding. After performing the SL-ARS system in an autism case, we found that autistic children performed well in understanding social situations and responding to other people’s physical behavior, as they can understand the sequence of social situations from SL-ARS, and observe the social reciprocity of social situations through the structured visual learning framework. In the process of playing, AR can improve the learning motivation and operating pleasure of autistic children to understand the social situation, and construct the relationship of social cognition and social reciprocity of autistic children into the context of social situations. This interactive learning gives them the social skills and context to develop social situations, and we believe that AR technology combined with the sequential social context is helpful to train autistic children’s social reciprocal ability.