TERL: Transformer Enhanced Reinforcement Learning for Relation Extraction

TERL: Transformer Enhanced Reinforcement Learning for Relation Extraction Yashen Wang China Academy of Electronics and Information Technology of CETC, Artificial Intelligence Institute of CETC yswang.arthur@gmail.com Tuo Shi Beijing Police College shituo@bjpc.edu.cn Xiaoye Ouyang National Engineering Laboratory for Risk Perception and Prevention (RPP), China Academy of Electronics and Information Technology ouyangxiaoye@cetc.com.cn Dayu Guo ∗ CETC Academy of Electronics and Information Technology Group Co.,Ltd. guodayu1@cetc.com.cn Abstract Relation Extraction (RE) task aims to discover the semantic relation that holds between two entities and contributes to many applications such as knowledge graph construction and completion. Reinforcement Learning (RL) has been widely used for RE task and achieved SOTA results, which are mainly designed with rewards to choose the optimal actions during the training procedure, to improve RE’s performance, especially for low-resource conditions. Recent work has shown that offline or online RL can be flexibly formulated as a sequence understanding problem and solved via approaches similar to large-scale pre-training language modeling. To strengthen the ability for understanding the semantic signals interactions among the given text sequence, this paper leverages Transformer architecture for RL-based RE methods, and proposes a generic framework called Transformer Enhanced RL (TERL) towards RE task. Unlike prior RL-based RE approaches that usually fit value functions or compute policy gradients, TERL only outputs the best actions by utilizing a masked Transformer. Experimental results show that the proposed TERL framework can improve many state-of-the-art RL-based RE methods. 1 Introduction Relation Extraction (RE) aims to discover the binary semantic relation between two entities in a sequence of words. E.g., given a sentence “· · · Carey will succeed Cathleen P. Black, who held the position for 15 years and will take on a new role as chairwoman of Hearst Magazines, the company said· · · ” (Xue et al., 2020), and we aim to predict the relation type between two entities “Cathleen P. Black” and “chairwoman” and the result is “per:title”. Deep neural network (DNN) driven methods have gained decent performance when labeled data is available (Hu et al., 2021b; Guo et al., 2020). While Reinforcement Learning (RL) based RE methods gain a lot of attention recently and show encouraging effects (Takanobu et al., 2018; Hu et al., 2021b; Wang and Zhang, 2021), especially in low-resource and few-shot conditions. Since this kinds of work requires fewer labeled data or could expand limited labeled data by exploiting information on unlabeled data to iteratively improve the performance (Hu et al., 2021b). Recent works have shown Transformers (Vaswani et al., 2017) can model high-dimensional distribu- tions of semantic concepts at scale, and several attempts have demonstrated the combination between transformers and RL architecture (Parisotto and Salakhutdinov, 2021; Parisotto et al., 2020; Zambaldi et al., 2019). These works have shown that the Transformer’s efficiency for modeling beneficial semantic interactions in the given sequence (Chen et al., 2021a; Zheng et al., 2022), which is very enlightening for RE task. Given the diversity of successful applications of such models (Chen et al., 2021a), this paper seeks to investigate their application to …

Content truncated to 500 words