Exploring Pair-Aware Triangular Attention for Biomedical Relation Extraction

Abstract

Biomedical relation extraction (BioRE) has become a research hotspot recently due to its crucial role in facilitating clinical diagnosis, treatment, and medical discovery. The advent of domain-specific language models, such as BioBERT and PubMedBERT customized for the biomedical domain, has revolutionized this task by fully learning contextualized entity representations and achieving remarkable performance. However, we argue that relying solely on entity-level modeling while neglecting pair-aware representations can lead to sub-optimal results, particularly in the complicated context of the biomedical literature. To address this issue, in this paper, we propose a novel Triangular Attention framework for Biomedical Relation Extraction (called TriA-BioRE) to comprehensively capture pair-aware representations in the biomedical domain. Specifically, we present a triangular attention module, including two triangular multiplications utilizing outgoing and incoming edges, and two triangular self-attention operations centered on the starting and ending nodes, respectively, together to enhance the pair-level modeling omnidirectionally for better BioRE performance. Extensive experiments on three biomedical datasets demonstrate that TriA-BioRE achieves substantially better results than its strong competitors in BioRE task. For reproducibility, our code and data are available at https://github.com/JasonCLEI/TriA-BioRE.