Generating Facial Expressions Associated with Text

Abstract

How will you react to the next post that you are going to read? In this paper we propose a learning system that is able to artificially alter the picture of a face in order to generate the emotion that is associated with a given input text. The face generation procedure is function of further information about the considered person, either given (topics of interest) or automatically estimated from the provided picture (age, sex). In particular, two Convolutional Networks are trained to predict age and sex, while two other Recurrent Neural Network-based models predict the topic and the dominant emotion in the input text. First Order Logic (FOL)-based functions are introduced to mix the outcome of the four neural models and to decide which emotion to generate, following the theory of T-Norms. Finally, the same theory is exploited to build a neural generative model of facial expressions, that is used create the final face. Experimental results are performed to assess the quality of the information extraction process and to show the outcome of the generative network.