Predicting Image Emotion Distribution by Learning Labels' Correlation

Abstract

Image emotion analysis attracts considerable attention with the increasing demanding of opinion mining in social networks. Emotion evoked by an image is always ambiguous for emotion's subjectivity. Different from previous researches on image emotion classification, Label Distribution Learning framework which assigns a set of labels with degree value to an instance, describes emotions more explicitly. However, in our study, we find that some labels have co-occurrence relation with others and all the labels together appear some structural forms. To make use of these relations as complementary information to the holistic distribution of labels, we analysis the correlations among emotion labels and then propose a method based on Structural Learning framework, which learns a mapping from images to the distribution labels with correlations. On the other hand, images usually contain some emotion-unrelated contents, to extract features that can represent image emotion at utmost, we propose a cropping method to select the emotional region from the images with the help of Fully Convolutional Networks. Extensive experiments on two widely used datasets show the advantages of our methods.