Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2737–2746 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2737 Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction Xuming Hu1, Chenwei Zhang2†, Yawen Yang1, Xiaohe Li1, Li Lin1, Lijie Wen1†, Philip S. Yu1,3 1Tsinghua University, 2Amazon, 3University of Illinois at Chicago 1{hxm19,yyw19,lixh18,lin-l16}@mails.tsinghua.edu.cn 1wenlj@tsinghua.edu.cn 2cwzhang@amazon.com 3psyu@uic.edu Abstract Low-resource Relation Extraction (LRE) aims to extract relation facts from limited labeled corpora when human annotation is scarce. Ex- isting works either utilize self-training scheme to generate pseudo labels that will cause the gradual drift problem, or leverage meta- learning scheme which does not solicit feed- back explicitly. To alleviate selection bias due to the lack of feedback loops in exist- ing LRE learning paradigms, we developed a Gradient Imitation Reinforcement Learning method to encourage pseudo label data to im- itate the gradient descent direction on labeled data and bootstrap its optimization capability through trial and error. We also propose a framework called GradLRE, which handles two major scenarios in low-resource relation extraction. Besides the scenario where unla- beled data is sufficient, GradLRE handles the situation where no unlabeled data is available, by exploiting a contextualized augmentation method to generate data. Experimental results on two public datasets demonstrate the effec- tiveness of GradLRE on low resource rela- tion extraction when comparing with baselines. Source code is available1. 1 Introduction Relation Extraction (RE) aims to discover the se- mantic relation that holds between two entities and transforms massive corpus into structured triplets (entityhead, relation, entitytail). For example, from “A letterhead was delivered to my officetail...", we can extract a relation Entity-Destination between head and tail entities. Neural RE meth- ods leverage high-quality annotated data or hu- man curated knowledge bases to achieve decent results (Zeng et al., 2017; Zhang et al., 2017). How- ever, these manually labeled data would be labor- intensive to obtain. This motivates a Low Resource 1https://github.com/THU-BPM/GradLRE †Corresponding Authors. !! !!" !# !# A letterhead was delivered to my officetail... !! : Entity-Destination !!" : Entity-Origin Figure 1: Gradient descent direction on labeled data (gl) and unlabeled data with correct or incorrect pseudo label (gu, g ′ u). Relation Extraction (LRE) task where annotations are scarce. Lots of efforts are devoted to improve the model generalization ability beyond learning directly from existing, limited annotations. Distant Supervision methods leverage facts stored in external knowl- edge bases (KBs) to obtain annotated triplets as the supervision (Mintz et al., 2009; Zeng et al., 2015). However, these methods should make a strong assumption that two co-occurring entities convey KB relations regardless of specific contexts, which makes model generate relations based on contextless rules and limits the generalization abil- ity. To leverage unlabeled data, Rosenberg et al. (2005) propose to assign pseudo labels on unla- beled data and leverage pseudo labels to iteratively improve the generalization capability of the model. However, during the training process, self-training models suffer from the gradual drift problem (Cur- ran et al., 2007; Zhang et al., 2016) caused …

Content truncated to 500 words