HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction

Findings of the Association for Computational Linguistics: ACL 2022, pages 2567 - 2578 May 22-27, 2022 c⃝2022 Association for Computational Linguistics HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction Dongyang Li1,2 ∗, Taolin Zhang3,4∗, Nan Hu1, Chengyu Wang3, Xiaofeng He1,5† 1 School of Computer Science and Technology, East China Normal University 2 Shanghai Key Laboratory of Trsustworthy Computing 3 Alibaba Group 4 School of Software Engineering, East China Normal University 5 Shanghai Research Institute for Intelligent Autonomous Systems dongyangli0612@gmail.com, zhangtl0519@gmail.com hunan.vinny1997@gmail.com, chengyu.wcy@alibaba-inc.com hexf@cs.ecnu.edu.cn Abstract Distant supervision assumes that any sentence containing the same entity pairs reflects iden- tical relationships. Previous works of distantly supervised relation extraction (DSRE) task gen- erally focus on sentence-level or bag-level de- noising techniques independently, neglecting the explicit interaction with cross levels. In this paper, we propose a Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction (HiCLRE) to reduce noisy sentences, which integrate the global structural information and local fine-grained interaction. Specifically, we propose a three-level hierarchi- cal learning framework to interact with cross levels, generating the de-noising context-aware representations via adapting the existing multi- head self-attention, named Multi-Granularity Recontextualization. Meanwhile, pseudo pos- itive samples are also provided in the spe- cific level for contrastive learning via a dy- namic gradient-based data augmentation strat- egy, named Dynamic Gradient Adversarial Per- turbation. Experiments demonstrate that Hi- CLRE significantly outperforms strong base- lines in various mainstream DSRE datasets.1 1 Introduction Relation extraction (RE) can draw relations of two entities from unstructured text. It can be widely used in natural language processing applications such as knowledge graph construction (Khatib et al., 2020; Tang et al., 2020) and question an- swering (Wang and Jiang, 2019; Liu et al., 2020; Saxena et al., 2020). Existing RE works (Wei et al., 2020; Alt et al., 2020; Veyseh et al., 2020) rely on a large-scale annotated dataset, which is time- consuming and labor-intensive. DSRE (Mintz et al., ∗D. Li and T. Zhang contributed equally to this work. †Corresponding author. 1The source code and data can be available at https://github. com/MatNLP/HiCLRE S1 e1 e2 en Cook Apple bag sentence entity S2 S3 /business /person /company America Washington /location /country /capital David Beckham Victoria /people /family /members B3 B4 : Cook Apple Steve Jobs · In 1998, Steve Jobs asked Cook to join Apple. · Under Cook’s leadership, Apple has increased its donations to charity. · The cook should put the apple into microwave on High for 2 minutes. e S1 S2 S3 B2 B1 /people /person /place_of_birth Larry Page Michigan Figure 1: Example of semantic relationships in specific levels and cross levels. The red cross means the seman- tic difference of two bag-level relations and the dotted arrow indicates the semantic overlapping of cross levels. (Best viewed in color). 2009) attempts to address this issue via automati- cally generating training text samples. Obviously, this assumption introduces noisy data and may hurt the performance. Hence, multi-instance learning (MIL) (Zeng et al., 2015) is further proposed to assign a bag containing “at …

Content truncated to 500 words