Image Captioning in Chinese and Its Application for Children with Autism Spectrum Disorder

Abstract

This research looks into the applications of image captioning in Chinese. In order to improve abilities of children with Autism Spectrum Disorder (ASD) in spontaneous language and turn-taking, during interacting with them, rehabilitation robots are used to track their attention and to describe attractive objects and scenes. This method has three advantages. First, the robots may attract the attention of children with ASD by describing the objects that of interest to them. Second, the robots may improve their ability of spontaneous language which is beneficial to enhancing superior language expressions and social communication with the robots or human beings. Third, describing objects that are interesting may develop the ability in picture description and may further improve their cognitive ability in rehabilitation training. The present study equips a socially interactive robot designed specifically for children with ASD with a dense image captioning approach. The contributions of this article are three-fold. First, an image captioning algorithm was developed to output Chinese description, whereas algorithms in previous studies were mostly catered for English description. Second, image captioning was adopted for the first time in the field of socially interactive robots. Third, our study demonstrates its potential in applying image captioning in robot-enhanced therapy, especially for Chinese children with ASD.