Comparative Analysis of different classifiers on crisis-related tweets: An elaborate study

Abstract

Twitter is a popular micro-blogging platform that has obtained a lot of reputation in the last few years and offer a diverse source of real-time information about different events, often during mass crises. During any crisis, it is necessary to filter through a huge amount of tweets rapidly to extract incident related information. Different machine learning (ML) algorithms have been used to classify crisis related tweets from non crisis-related ones, and has great importance in constructing an emergency management framework. These algorithms rely heavily on datasets used, and also different hyper-parameters which need to be tuned to provide better performance. Hence, this paper focuses on: (1) different Natural Language Processing (NLP) techniques to make tweets suitable for applying ML algorithms, (2) hyper-parameter tuning of neural networks when used as classifiers on short messages, tweets, (3) comparative analysis of different state-of-the-art ML algorithms (classifiers) which can be applied to categorize crisis-related tweets with a higher accuracy. The experiments have been done on six different crisis related datasets, each approximately consisting of 10,000 tweets. Analysis have shown that Support Vector Machines and Logistic Regression have performed significantly well than Naive Bayes and Neural Networks (NN) with a very high accuracy of 96% (variations seen with different dataset though). With proper hyper-parameter tuning, NN have also showed promising results.