TAG Parsing with Neural Networks and Vector Representations of Supertags

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1712–1722 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics TAG Parsing with Neural Networks and Vector Representations of Supertags Jungo Kasai Dept. of Computer Science Yale University jungo.kasai@yale.edu Robert Frank Dept. of Linguistics Yale University robert.frank@yale.edu R. Thomas McCoy Dept. of Linguistics Yale University richard.mccoy@yale.edu Owen Rambow DSI Columbia University rambow@ccls.columbia.edu Alexis Nasr LIF Universit´e Aix Marseille Alexis.Nasr@lif.univ-mrs.fr Abstract We present supertagging-based models for Tree Adjoining Grammar parsing that use neural network architectures and dense vector representation of supertags (ele- mentary trees) to achieve state-of-the-art performance in unlabeled and labeled at- tachment scores. The shift-reduce pars- ing model eschews lexical information en- tirely, and uses only the 1-best supertags to parse a sentence, providing further support for the claim that supertagging is “almost parsing.” We demonstrate that the embed- ding vector representations the parser in- duces for supertags possess linguistically interpretable structure, supporting analo- gies between grammatical structures like those familiar from recent work in distri- butional semantics. This dense represen- tation of supertags overcomes the draw- backs for statistical models of TAG as compared to CCG parsing, raising the pos- sibility that TAG is a viable alternative for NLP tasks that require the assignment of richer structural descriptions to sentences. 1 Introduction Recent work has applied Combinatory Categorial Grammar (CCG, Steedman and Baldridge (2011)) to the problem of broad-coverage parsing in order to derive grammatical representations that are suf- ficiently rich to support tasks requiring deeper rep- resentation of a sentence’s meaning (Lewis et al., 2015; Reddy et al., 2016; Nadejde et al., 2017). Yet CCG is only one of a number of mildly context-sensitive grammar formalisms that can provide such rich representations, and each has distinct advantages. In this paper we explore the applicability of another formalism, Tree Adjoin- ing Grammar (TAG, Joshi and Schabes (1997)), to the task of broad-coverage parsing. TAG and CCG share the property of lexicaliza- tion: words are associated with elementary units of grammatical structure which are composed dur- ing a derivation using one of a small set of oper- ations to produce a parse tree. The task of pars- ing involves the construction of a derivation tree that encodes the application of this set of actions to a set of elementary lexically-associated objects. TAG differs from CCG in having an even richer set of lexical units, so that the identification of these units in a derivation could be even more informa- tive for subsequent tasks involving semantic inter- pretation, translation and the like, which have been the focus of CCG-based work. The elementary units of CCG and TAG (cate- gories for CCG, and elementary trees for TAG) determine a word’s combinatory potential, in a way that is not the case for the usual part-of- speech tags used in parsing. Indeed, the assign- ment of elementary objects to the words in a sen- tence almost determines the possible parse for a sentence. The near uniqueness of a parse …

Content truncated to 500 words