An Attention-Based ID-CNNs-CRF Model for Named Entity Recognition on Clinical Electronic Medical Records

Abstract

Named Entity Recognition (NER) on Clinical Electronic Medical Records (CEMR) is a fundamental step in extracting disease knowledge by identifying specific entity terms such as diseases, symptoms, etc. However, the state-of-the-art NER methods based on Long Short-Term Memory (LSTM) fail to fully exploit GPU parallelism under the massive medical records. Although a novel NER method based on Iterated Dilated CNNs (ID-CNNs) can accelerate network computing, it tends to ignore the word-order feature and semantic information of the current word. In order to enhance the performance of ID-CNNs-based models on NER tasks, an attention-based ID-CNNs-CRF model which combines word-order feature and local context is proposed. Firstly, Position Embedding is utilized to fuse word-order information. Secondly, ID-CNNs architecture is used to rapidly extract global semantic information. Simultaneously, the attention mechanism is employed to pay attention to the local context. Finally, we apply the CRF to obtain the optimal tag sequence. Experiments conducted on two CEMR datasets show that our model outperforms traditional ones. The F1-scores of 94.55% and 91.17% are obtained respectively on these two datasets and both are better than LSTMs-based models.