Emoji Prediction: A Transfer Learning Approach

Abstract

We present a transfer learning model for the Emoji Prediction task described at SemEval-2018 Task 2. Given a text of tweet, the task aims to predict the most likely emoji to be used within such tweet. The proposed method used a pre-training and fine-tuning strategy, which applies the pre-learned knowledge from several upstream tasks to downstream Emoji Prediction task, solving the data scarcity issue suffered by most of the SemEval-2018 participants using supervised learning strategy. Our transfer learning-based model can outperform state-of-the-art system (best performer at SemEval-2018) by 2.53% in macro F-score. Except from providing details of our system, this paper also intends to provide a comparison between supervised learning models and transfer learning models in solving Emoji Prediction task.