Towards Relation Extraction from Speech

Abstract

Relation extraction has focused on extracting semantic relationships between entities from the unstructured written textual data. However, with the vast and rapidly increasing amounts of spoken data, relation extraction from speech is an important but under-explored problem. In this paper, we propose a new information extraction task, speech relation extraction (SpeechRE). To facilitate further research, we construct the first synthetic training datasets, as well as the first human-spoken test set with native English speakers. We establish strong baseline performance for SpeechRE via two approaches. The pipeline approach connects a pretrained ASR module with a text-based relation extraction module. The end-to-end approach employs a cross-modal encoder-decoder architecture. Our comprehensive experiments reveal the relative strengths and weaknesses of these approaches, and shed light on important future directions in SpeechRE research. We share the source code and datasets on https://github.com/wutong8023/SpeechRE.