Dynamic Graph Transformer for Implicit Tag Recognition

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1426–1431 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1426 Dynamic Graph Transformer for Implicit Tag Recognition Yi-Ting Liou,1 Chung-Chi Chen,1 Hen-Hsen Huang,2,3 Hsin-Hsi Chen1,3 1 Department of Computer Science and Information Engineering National Taiwan University, Taiwan 2 Department of Computer Science, National Chengchi University, Taiwan 3 MOST Joint Research Center for AI Technology and All Vista Healthcare, Taiwan ytliou, cjchen@nlg.csie.ntu.edu.tw hhhuang@nccu.edu.tw, hhchen@ntu.edu.tw Abstract Textual information extraction is a typical re- search topic in the NLP community. Several NLP tasks such as named entity recognition and relation extraction between entities have been well-studied in previous work. However, few works pay their attention to the implicit information. For example, a financial news ar- ticle mentioned “Apple Inc.” may be also re- lated to Samsung, even though Samsung is not explicitly mentioned in this article. This work presents a novel dynamic graph transformer that distills the textual information and the en- tity relations on the fly. Experimental results confirm the effectiveness of our approach to implicit tag recognition. 1 Introduction Documents on the web deliver and spread lots of most recent information to people worldwide. In order to automatically update the real-world infor- mation, textual information extraction is a funda- mental issue for NLP researchers. Many down- stream tasks such as fake news detection (Wu et al., 2019) and stock movement prediction (Peng and Jiang, 2016) can benefit from the extracted infor- mation. However, most of the previous works (Hu et al., 2018; Xu and Cohen, 2018) focus on using explicit information in articles and do not consider the implicit information. For example, two compa- nies, Sprint and T-Mobile, are explicitly mentioned companies in the news article in Figure 1. How- ever, the stock price of a non-mentioned company, SoftBank, may be influenced since Softbank owns shares of Sprint. Although this kind of inference is intuitive for professional analysts, few previous works take such implicit information into consider- ation. In this paper, we aim to increase the sense of machines toward this kind of implicit information. Transformer-based (Vaswani et al., 2017) neural networks achieve state-of-the-art performances in Title: The Judge Nodded! Sprint Rise 70% in After-hour Trading. News Article: A federal judge gave his blessing to the US$26.5 billion merger between T-Mobile and Sprint on Feb. 11, several months after the deal got final antitrust approval from the U.S. government. Sprint surges 68.75%. T-Mobile rise 7.36%. Related Stock: Sprint, T-Mobile, SoftBank Figure 1: An example of the implicit information in news articles. many NLP tasks (Devlin et al., 2018; Malmi et al., 2019). To model the relationships between enti- ties, graph neural network (GNN) is a well-known architecture for representing the knowledge and additional information (Fu et al., 2019; You et al., 2020). Furthermore, the models blending these two architectures show their effectiveness (Lu et al., 2020). In this paper, we propose dynamic graph transformer (DGT), a novel blend of Transformer and GNN. In …

Content truncated to 500 words