An Empirical Study on Joint Entities-relations Extraction of Chinese Text Based on BERT

Abstract

Joint Entities-relations extraction (JERE), which can solve the error propagation problem in piped-line extraction, uses a single model to combine the two tasks by different strategies. The neural network-based methods for the joint extraction show better performances than the state-of-the-art results, but the constructed model are complicated. hyper-parameters selection and model training are also difficult. For simplifying the model construction and improving the accuracy of extraction, we transferred the joint extraction problem to a sequential label problem formally, combined different strategies in Encoding layer and Decoding Layer of end-to-end model, focused on the using of Bert on Encoding layer with different tagging scheme, then designed the comparative experiments on Chinese text dataset, the empirical results showed that 1)Using Bert as the embedding layer of the model is better than word2vec, but not so significant. 2)The final results of Bert followed by complex neural net-works such as BiLSTM and simple neural networks such as MLP on experiments are very close, which means Bert can simplify the model construction, it also means that we can get a good result with less training time by a simple neural networks. 3)The new tagging scheme is better than the BIO tagging scheme, and if it is combined with the pre-training language model, we can quickly build a simple and sufficiently accurate model on JERE tasks.