Psychomime Classification and Visualization Using a Self-Organizing Map for Implementing Emotional Spoken Dialogue System

Abstract

We aim to automatically and appropriately classify Japanese psycho-mimes that represent users’ emotional aspects, such as “pokapoka” and “tikutiku,” and visualize the classification results as a map by using a self-organizing map (SOM) algorithm as the basis of the implementing a spoken dialogue application with emotional agents. Dealing with psychomimes and visualizing their classification has become increasingly important because they reflect the speaker’s emotions and frequently appear in communication, particularly in Japanese. However it is difficult to communicate meaning to people who do not understand psychomimes because they are not directly perceivable. We experimentally classified psychomimes with SOM and represented the results as maps with significant three-vector dimensions, i.e., three verb classes assigned by a verb thesaurus dictionary. The experimental results demonstrated that our method was effective in accomplishing a precision rate for classification that was higher than 80% in almost every group attained by setting an adequate threshold according to the distribution of the SOM node data. Moreover, we demonstrated the importance of selecting not one or two verb classes but three to depict the classification results as a map that can express subtle differences in emotion.