Technology Trend Prediction from Social Media using Long Short Term Memory Network

Abstract

The present study aims to develop a deep learning based data analysis model which would act as a framework for efficient implementation of Social Media Analysis (SMA). This framework combines four process for data analysis i) Data collection, ii) Pre-processing, iii) Technology Classification and iv) Technology Trend prediction. The study uses hybrid model of Deep Feed Forward Neural Network (DNN) and Long Short Term Memory (LSTM) network. The technology and tools learned by people becomes obsolete in short period of time. As I.T. industry required frequent upgrades in knowledge and new technologies are being released, it is essential to track and know new upcoming technology trend of the field. To achieve this aim deep learning model is developed to identify upcoming technologies from social media threads. This paper presents LSTM trend prediction model to predict technology trends from unstructured text content of social media sources. The proposed method ensembles classification and regression process in single architecture. First, it uses deep learning algorithm to build a classifier to correctly predict the technology topic of a discussion thread from its description. After technology identification, how frequently that technology is discussed with respect to time is calculated to generate temporal series of frequencies. The Long Short Term Memory (LSTM) network is combined with Deep Feed Forward Neural Network model for processing temporal topic sequence and frequencies recursively to predict technology trends from of social posts generated on social platforms.