Entity Relative Position Representation based Multi-head Selection for Joint Entity and Relation Extraction

Entity Relative Position Representation based Multi-head Selection for Joint Entity and Relation Extraction Tianyang Zhao1∗and Zhao Yan2 and Yunbo Cao2 and Zhoujun Li1 1State Key Lab of Software Development Environment, Beihang University, Beijing, China 2Tencent Cloud Xiaowei, Beijing, China {tyzhao,lizj}@buaa.edu.cn, {zhaoyan,yunbocao}@tencent.com Abstract Joint entity and relation extraction has received increasing interests recently, due to the capa- bility of utilizing the interactions between both steps. Among existing studies, the Multi-Head Selection (MHS) framework is efficient in extracting entities and relations simultaneously. How- ever, the method is weak for its limited performance. In this paper, we propose several effective insights to address this problem. First, we propose an entity-specific Relative Position Repre- sentation (eRPR) to allow the model to fully leverage the distance information between entities and context tokens. Second, we introduce an auxiliary Global Relation Classification (GRC) to enhance the learning of local contextual features. Moreover, we improve the semantic repre- sentation by adopting a pre-trained language model BERT as the feature encoder. Finally, these new keypoints are closely integrated with the multi-head selection framework and optimized jointly. Extensive experiments on two benchmark datasets demonstrate that our approach over- whelmingly outperforms previous works in terms of all evaluation metrics, achieving significant improvements for relation F1 by +2.40% on CoNLL04 and +1.90% on ACE05, respectively. 1 Introduction The entity-relation extraction task aims to recognize the entity spans from a sentence and detect the relations holds between two entities. Generally, it can be formed as extracting triplets (e1, r, e2), which denotes that the relation r holds between the head entity e1 and the tail entity e2, i.e., (John Smith, Live-In, Atlanta). It plays a vital role in the information extraction area and has attracted increasing attention in recent years. Traditional pipelined methods divide the task into two phases, named entity recognition (NER) and relation extraction (RE) (Miwa et al., 2009; Chan and Roth, 2011; Lin et al., 2016). As such methods neglect the underlying correlations between the two phases and suffer from the error propagation issue, recent works propose to extract entities and relations jointly. These joint models fall into two paradigms. The first paradigm can be denoted as (e1, e2) → r, which first recognizes all entities in the sentence, then classifies the relation depend on each extracted entity pairs. However, these methods require enu- merating all possible entity pairs and the relation classification may be affected by the redundant ones. While another paradigm is referred as e1 → (r, e2), which detects head entities first and then predicts the corresponding relations and tail entities (Bekoulis et al., 2018; Li et al., 2019; Zhao et al., 2020). Com- paring with the first paradigm, the second one can jointly identify entities and all the possible relations between them at once. A typical approach is the Multi-Head Selection (MHS) framework (Bekoulis et al., 2018). It first recognizes head entities using the BiLSTM-CRF structure and then performs tail entity extraction and relation extraction in one pass based on multiclass classification. The advantage of the …

Content truncated to 500 words