MEDeep: A Deep Learning Based Model for Memotion Analysis

Abstract

A meme is an idea or an expression that becomes a trend and usually spreads within a culture through imitation, carrying a significant meaning representing a specific concept or theme. It represents a particular message by the combination of image and text. An enormous number of memes are spread every day via social media platforms to share sarcastic, humorous, and offensive messages. Therefore, to control the spread of offensive language and propaganda, there is a need for an automated mechanism to classify the meme. Considering this, the proposed study aimed to investigate the impact of using different data modality i.e., text, images, combined (text and images) for memotion analysis using Deep Learning (DL) models. Several pre-trained DL models such as ResNet152V2, VGG19, EfficientNetB7 were used for the images. While, Convolutional Neural Network (CNN) and CNN+ Long short-term memory (LSTM) were implemented for the text classification. Experimental results reveal that the CNN model outperformed for the text, and EfficientNetB7 achieved the best performance on the images. However, for the multimodal analysis, early fusion technique was used and classification was performed using CNN and EfficientNetB7 model. The study found that Glove embedding and CNN model using text produced the highest results among all the experiments conducted. The model achieved an accuracy, F1-macro, precision, and recall of 0.8387, 0.8352, 0.8361, 0.8487, respectively. The results exhibit that the proposed model outperforms other baseline studies.