A Multi-Gate Encoder for Joint Entity and Relation Extraction

A Multi-Gate Encoder for Joint Entity and Relation Extraction Xiong Xiong1,2, Yunfei Liu1,2, Anqi Liu1, Shuai Gong1,2, Shengyang Li1,2∗ 1Key Laboratory of Space Utilization, Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, Beijing, China 2University of Chinese Academy of Sciences, Beijing, China {xiongxiong20,liuyunfei,liuaq,gongshuai19,shyli}@csu.ac.cn Abstract Named entity recognition and relation extraction are core sub-tasks of relational triple extraction. Recent studies have used parameter sharing or joint decoding to create interaction between these two tasks. However, ensuring the specificity of task-specific traits while the two tasks interact properly is a huge difficulty. We propose a multi-gate encoder that models bidirectional task interaction while keeping sufficient feature specificity based on gating mechanism in this paper. Precisely, we design two types of independent gates: task gates to generate task-specific features and interaction gates to generate instructive features to guide the opposite task. Our experiments show that our method increases the state-of-the-art (SOTA) relation F1 scores on ACE04, ACE05 and SciERC datasets to 63.8% (+1.3%), 68.2% (+1.4%), 39.4% (+1.0%), respectively, with higher inference speed over previous SOTA model. 1 Introduction Extracting relational facts from unstructured texts is a fundamental task in information extraction. This task can be decomposed into two sub-tasks: Named Entity Recognition (NER) (Florian et al., 2003), which aims to recognize the boundaries and types of entities; and Relation Extraction (RE) (Zelenko et al., 2002), which aims to extract semantic relations between entities. The extracted relational triples in the form of (subject, relation, object) are basic elements of large-scale knowledge graphs (Lin et al., 2015). Traditional approaches perform NER and RE in a pipelined fashion (Zhou et al., 2005; Chan and Roth, 2011; Gormley et al., 2015). They first extract all the entities in a given text, and then identify pairwise relations between the extracted entities. However, because the two sub-tasks are modeled independently, pipelined methods are vulnerable to error propagation issue. Since the interaction between NER and RE is neglected, the errors accumulated in the previous NER stage cannot be corrected in the subsequent RE stage. To resolve this issue, some joint models have been proposed to model these two tasks simultaneously. Early feature-based joint models (Yu and Lam, 2010; Miwa and Sasaki, 2014) rely on complicated feature engineering to build interaction between entities and relations. More recently, neural joint models have attracted increasing research interest and have demonstrated promising performance on joint entity and relation extraction. In existing neural joint models, there are mainly two ways to build the interaction between NER and RE: parameter sharing and joint decoding. In parameter sharing methods (Zeng et al., 2018; Bekoulis et al., 2018a; Dixit and Al-Onaizan, 2019), NER model and RE model are built on top of a shared encoding layer to achieve joint learning. However, approaches based on parameter sharing implicitly rather than explicitly model the inter-task interaction, leading to insufficient excavation of the inherent association between the two tasks. Moreover, these two tasks focus on different contextual information (Zhong and Chen, 2021; Ye et al., 2022), but methods …

Content truncated to 500 words