Enhancing Ontology Knowledge for Domain-Specific Joint Entity and Relation Extraction

Enhancing Ontology Knowledge for Domain-Specific Joint Entity and Relation Extraction Xiong Xiong1,2, Chen Wang1,2, Yunfei Liu1,2, Shengyang Li1,2∗ 1Key Laboratory of Space Utilization, Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, Beijing, China 2University of Chinese Academy of Sciences, Beijing, China {xiongxiong20,wangchen21,liuyunfei,shyli}@csu.ac.cn Abstract Pre-trained language models (PLMs) have been widely used in entity and relation extraction methods in recent years. However, due to the semantic gap between general-domain text used for pre-training and domain-specific text, these methods encounter semantic redundancy and domain semantics insufficiency when it comes to domain-specific tasks. To mitigate this issue, we propose a low-cost and effective knowledge-enhanced method to facilitate domain-specific semantics modeling in joint entity and relation extraction. Precisely, we use ontology and entity type descriptions as domain knowledge sources, which are encoded and incorporated into the downstream entity and relation extraction model to improve its understanding of domain-specific information. We construct a dataset called SSUIE-RE for Chinese entity and relation extraction in space science and utilization domain of China Manned Space Engineering, which contains a wealth of domain-specific knowledge. The experimental results on SSUIE-RE demonstrate the effectiveness of our method, achieving a 1.4% absolute improvement in relation F1 score over previous best approach. 1 Introduction Extracting relational triples from plain text is a fundamental task in information extraction and it’s an essential step in knowledge graph (KG) construction (Lin et al., 2015). Traditional methods perform Named Entity Recognition (NER) and Relation Extraction (RE) in a pipelined manner, that is, first extract entities, and then perform relation classification on entity pairs (Zhou et al., 2005; Chan and Roth, 2011; Gormley et al., 2015). However, since the entity model and relation model are modeled separately, pipelined methods suffer from the problem of error propagation. To address this issue, some joint methods have been proposed (Yu and Lam, 2010; Li and Ji, 2014; Zheng et al., 2017; Wang and Lu, 2020; Yan et al., 2021; Xiong et al., 2022). The task of joint entity and relation extraction aims to simultaneously conduct entity recognition and relation classification in an end-to-end manner. In recent years, with the development of pre-trained language models (PLMs) such as BERT (Devlin et al., 2019) and GPT (Radford et al., 2018), many entity and relation extraction methods have adopted the paradigm of pre-training and fine-tuning. They utilize PLMs to encode the contextual representations of input text and design various downstream models for task-specific fine-tuning. However, when employed for domain-specific entity and relation extraction, this paradigm suffers from problems of semantic redundancy and insufficiency of domain-specific semantics, particularly in highly specialized domains. On the one hand, PLMs are usually trained on general-domain corpora, which results in a significant amount of redundant semantic information that may not be relevant to specific domains and a lack of sufficient domain-specific semantic information. On the other hand, modeling domain-specific information in this paradigm depends primarily on the role of downstream model and domain-specific labeled data in the fine-tuning stage. However, due to the significantly smaller parameter …

Content truncated to 500 words