Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1351–1360 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1351 Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data Wei Ye1*†, Bo Li1,3*, Rui Xie1,2, Zhonghao Sheng1,2, Long Chen1,2 and Shikun Zhang1 1National Engineering Research Center for Software Engineering, Peking University 2School of Software and Microelectronics, Peking University 3Automation Dept, Beijing University of Posts and Telecommunications. wye@pku.edu.cn, deepblue.lb@gmail.com, {ruixie, zhonghao.sheng, clcmlxl, zhangsk}@pku.edu.cn Abstract In practical scenario, relation extraction needs to first identify entity pairs that have relation and then assign a correct relation class. How- ever, the number of non-relation entity pairs in context (negative instances) usually far ex- ceeds the others (positive instances), which negatively affects a model’s performance. To mitigate this problem, we propose a multi- task architecture which jointly trains a model to perform relation identification with cross- entropy loss and relation classification with ranking loss. Meanwhile, we observe that a sentence may have multiple entities and re- lation mentions, and the patterns in which the entities appear in a sentence may contain useful semantic information that can be uti- lized to distinguish between positive and neg- ative instances. Thus we further incorporate the embeddings of character-wise/word-wise BIO tag from the named entity recognition task into character/word embeddings to enrich the input representation. Experiment results show that our proposed approach can signifi- cantly improve the performance of a baseline model with more than 10% absolute increase in F1-score, and outperform the state-of-the- art models on ACE 2005 Chinese and English corpus. Moreover, BIO tag embeddings are particularly effective and can be used to im- prove other models as well. 1 Introduction Relation extraction, which aims to extract seman- tic relations from a given instance—entity pair and the corresponding text in context, is an important and challenging task in information extraction. It serves as a step stone for many downstream tasks such as question answering and knowledge graph construction. * indicates equal contribution. † Corresponding author. Traditionally, researchers mainly use ei- ther feature-based methods (Kambhatla, 2004; Boschee et al., 2005; GuoDong et al., 2005; Jiang and Zhai, 2007; Chan and Roth, 2010; Sun et al., 2011; Nguyen and Grishman, 2014) or kernel- based methods (Zelenko et al., 2003; Culotta and Sorensen, 2004; Bunescu and Mooney, 2005; Mooney and Bunescu, 2006; Zhang et al., 2006; Zhou et al., 2007; Giuliano et al., 2007; Qian et al., 2008; Nguyen et al., 2009; Sun and Han, 2014) for relation extraction, which tend to heav- ily rely on handcraft features and existing natural language processing (NLP) tools. Recently, deep learning models, including convolutional neural network (CNN) (Liu et al., 2013; Zeng et al., 2014; Nguyen and Grishman, 2015; Zeng et al., 2015; dos Santos et al., 2015; Lin et al., 2016) and recurrent neural network (RNN) (Miwa and Bansal, 2016; Zhou et al., 2016; She et al., 2018) w/o variants of attention mechanism have been widely applied to relation …

Content truncated to 500 words