Query-Based Keyphrase Extraction from Long Documents

Abstract

Transformer-based architectures in natural language processing force input size limits that can be problematic when long documents need to be processed. This paper overcomes thiissue for keyphrase extraction by chunking the long documents while keeping a global context as a query defining thtopic for which relevant keyphrases should be extracted. Thdeveloped system employs a pre-trained BERT model and adapts it to estimate the probability that a given text span forms a keyphrase. We experimented using various contexsizes on two popular datasets, Inspec and SemEval, and large novel dataset. The presented results show that a shortecontext with a query overcomes a longer one without thquery on long documents.1.