Hierarchical Pointer Net Parsing

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1007–1017, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1007 Hierarchical Pointer Net Parsing Linlin Liu12∗†, Xiang Lin1†, Shafiq Joty13, Simeng Han1, Lidong Bing2 1Nanyang Technological University, Singapore 2R&D Center Singapore, Machine Intelligence Technology, Alibaba DAMO Academy 3Salesforce Research Asia, Singapore {linx0057, srjoty, hans0035}@ntu.edu.sg {linlin.liu, l.bing}@alibaba-inc.com Abstract Transition-based top-down parsing with pointer networks has achieved state-of-the-art results in multiple parsing tasks, while having a linear time complexity. However, the decoder of these parsers has a sequential structure, which does not yield the most appropriate inductive bias for deriving tree structures. In this paper, we propose hierarchi- cal pointer network parsers, and apply them to dependency and sentence-level discourse parsing tasks. Our results on standard bench- mark datasets demonstrate the effectiveness of our approach, outperforming existing methods and setting a new state-of-the-art. 1 Introduction Parsing of sentences is a core natural language un- derstanding task, where the goal is to construct a tree structure that best describes the relation- ships between the tree constituents (e.g., words, phrases). For example, Figure 1 shows examples of a dependency tree and a sentence-level dis- course tree that respectively represent how the words and clauses are related in a sentence. Such parse trees are directly useful in numerous NLP applications, and also serve as intermediate rep- resentations for further language processing tasks such as semantic and discourse processing. Existing approaches to parsing can be distin- guished based on whether they employ a greedy transition-based algorithm (Marcu, 1999; Zhang and Nivre, 2011; Wang et al., 2017) or a glob- ally optimized algorithm such as graph-based methods for dependency parsing (Eisner, 1996) or chart parsing for discourse (Soricut and Marcu, 2003; Joty et al., 2015). Transition-based parsers build the tree incrementally by making a series ∗ Linlin Liu is under the Joint PhD Program between Alibaba and Nanyang Technological University. † Equal contribution. Figure 1: (a) A dependency tree for a sentence; (b) a discourse tree for the sentence “[Now that’s name- dropping,]e1 [if you know]e2 [what I mean.]e3”, where ‘S’ denotes Satellite and ‘N’ denotes Nucleus. of shift-reduce decisions. The advantage of this method is that the parsing time is linear with re- spect to the sequence length. The limitation, how- ever, is that the decisions made at each step are based on local information, disallowing the model to capture long distance dependencies and also causing error propagation to subsequent steps. Re- cent methods attempt to address this issue us- ing neural networks capable of remembering long range relationships such as Stacked LSTMs (Dyer et al., 2015; Ballesteros et al., 2015) or using glob- ally normalized models (Andor et al., 2016). The globally optimized methods, on the other hand, learn scoring functions for subtrees and per- form search over all possible trees to find the most probable tree for a text. Recent graph-based meth- ods use neural models as scoring …

Content truncated to 500 words