Multi-applicable text classification based on deep neural network

Abstract

Most long text classification methods based on deep learning have problems such as semantics sparsity and long-distance dependence. To tackle these problems, a novel multi-applicable text classification based on deep neural network (MTDNN) is proposed, which 278 J. Yang et al. contains a bidirectional encoder representation from transformer (BERT), a dimension reduction layer, and the bidirectional long short-term memory (Bi-LSTM) combining attention mechanism. BERT is used to pre-train the words into the word embedding vectors. The dimension reduction layer extracts the feature phrase representations with higher weight from the word embedding vectors. The Bi-LSTM captures both the forward and backward context representations. An attention mechanism is employed to focus on the information outputted from the Bi-LSTM. The experimental results illustrate that the accuracy of the MTDNN for long text, short text classification, and sentiment analysis reaches 94.95%, 93.53% and 92.32%, respectively. The results show that our method outperforms the other state-of-the-art text classification methods.