Intelligent educational system for autistic children using augmented reality and machine learning

Abstract

Autism is a severe disorder affecting 1 in 160 children globally. Autism comprises of several development disabilities such as social, communicational and behavioural challenges. Children being diagnosed by the autism mainly face a hard time studying curriculum in inclusive classrooms based on their IQ level and the autism levels. Although, different strategies and learning teaching tools are available to support autistic children, only few systems aid them in learning efficiently, and are not highly interactive. Thus, the proposed Intelligent Education System primarily focuses on providing interactive learning experience to the autistic children with IQ level >50% and efficient teaching assistance to their tutors using augmented reality and machine learning in both English and Arabic The capability of the education system to perform an action, allows the autistic child to interact with the playable sand and gain interest. In learning stage, once the child scribbles on the sandbox, Kinect 3D camera captures and recognizes the drawn image. After the refinement and recognition of the image using OpenCV and classification model, the stored set of real world object are projected on the canvas. Besides, a webcam captures the facial expression of the child, and emotion detection algorithm determines the reaction of the child. Based on the child’s emotion, the current object is projected and pronounced three times to enforce better learning. Once the instructor chooses the language and character to be taught using the developed mobile application, the system displays it over the sandbox and further three objects that starts with the particular character are pronounced and projected. The system is tested rigorously with large set of users, and the results prove the efficiency of the system and happiness of the autistic children in better learning.