Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1371–1377 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1371 Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers Haoyu Wang∗† Ming Tan∗† Mo Yu∗‡ Shiyu Chang‡ Dakuo Wang‡ Kun Xu§ Xiaoxiao Guo‡ Saloni Potdar† †IBM Watson ‡IBM Research §Tencent AI Lab Abstract The state-of-the-art solutions for extracting multiple entity-relations from an input para- graph always require a multiple-pass encoding on the input. This paper proposes a new so- lution that can complete the multiple entity- relations extraction task with only one-pass encoding on the input corpus, and achieve a new state-of-the-art accuracy performance, as demonstrated in the ACE 2005 benchmark. Our solution is built on top of the pre-trained self-attentive models (Transformer). Since our method uses a single-pass to compute all rela- tions at once, it scales to larger datasets easily; which makes it more usable in real-world ap- plications. 1 1 Introduction Relation extraction (RE) aims to find the semantic relation between a pair of entity mentions from an input paragraph. A solution to this task is essential for many downstream NLP applications such as automatic knowledge-base completion (Surdeanu et al., 2012; Riedel et al., 2013; Verga et al., 2016), knowledge base question answering (Yih et al., 2015; Xu et al., 2016; Yu et al., 2017), and symbolic approaches for visual question answer- ing (Mao et al., 2019; Hu et al., 2019), etc. One particular type of the RE task is multiple- relations extraction (MRE) that aims to recognize relations of multiple pairs of entity mentions from an input paragraph. Because in real-world appli- cations, whose input paragraphs dominantly con- tain multiple pairs of entities, an efficient and ef- fective solution for MRE has more important and more practical implications. However, nearly all existing approaches for MRE tasks (Qu et al., ∗ Equal contributions from the corresponding authors: {wanghaoy,mingtan,yum}@us.ibm.com. Part of work was done when Kun was at IBM. 1https://github.com/helloeve/mre-in-one-pass. ×12 … … Entity-aware Self-attention+ Feed-forward … in south suburbs of Bagh #dad and Iraqi artillery fired … Linear PART-WHOLE(e1,e2) ART(e1,e2) Pool Pool Linear Figure 1: Model Architecture. Different pairs of entities, e.g., (Iraqi and artillery), (southern suburbs, Baghdad) are predicted simultaneously. 2014; Gormley et al., 2015; Nguyen and Grish- man, 2015) adopt some variations of the single- relation extraction (SRE) approach, which treats each pair of entity mentions as an independent in- stance, and requires multiple passes of encoding for the multiple pairs of entities. The drawback of this approach is obvious – it is computationally ex- pensive and this issue becomes more severe when the input paragraph is large, making this solution impossible to implement when the encoding step involves deep models. This work presents a solution that can resolve the inefficient multiple-passes issue of existing so- lutions for MRE by encoding the input only once, which significantly increases the efficiency and scalability. Specifically, the proposed solution is built on top of the existing transformer-based, pre- trained general-purposed language encoders. …

Content truncated to 500 words