Neural Shift-Reduce CCG Semantic Parsing

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1775–1786, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Neural Shift-Reduce CCG Semantic Parsing Dipendra K. Misra and Yoav Artzi Department of Computer Science and Cornell Tech Cornell University New York, NY 10011 {dkm,yoav}@cs.cornell.edu Abstract We present a shift-reduce CCG semantic parser. Our parser uses a neural network ar- chitecture that balances model capacity and computational cost. We train by transferring a model from a computationally expensive log- linear CKY parser. Our learner addresses two challenges: selecting the best parse for learn- ing when the CKY parser generates multiple correct trees, and learning from partial deriva- tions when the CKY parser fails to parse. We evaluate on AMR parsing. Our parser per- forms comparably to the CKY parser, while doing significantly fewer operations. We also present results for greedy semantic parsing with a relatively small drop in performance. 1 Introduction Shift-reduce parsing is a class of parsing methods that guarantees a linear number of operations in sen- tence length. This is a desired property for practical applications that require processing large amounts of text or real-time response. Recently, such techniques were used to build state-of-the-art syntactic parsers, and have demonstrated the effectiveness of deep neural architectures for decision making in linear- time dependency parsing (Chen and Manning, 2014; Dyer et al., 2015; Andor et al., 2016; Kiperwasser and Goldberg, 2016). In contrast, semantic parsing often relies on algorithms with polynomial number of operations, which results in slow parsing times unsuitable for practical applications. In this paper, we apply shift-reduce parsing to semantic parsing. Specifically, we study transferring a learned Combi- natory Categorial Grammar (CCG; Steedman, 1996, 2000) from a dynamic-programming CKY model to a shift-reduce neural network architecture. We focus on the feed-forward architecture of Chen and Manning (2014), where each parsing step is a multi-class classification problem. The state of the parser is represented using simple feature em- beddings that are passed through a multilayer per- ceptron to select the next action. While simple, the capacity of this model to capture interactions be- tween primitive features, instead of relying on sparse complex features, has led to new state-of-the-art per- formance (Andor et al., 2016). However, applying this architecture to semantic parsing presents learn- ing and inference challenges. In contrast to dependency parsing, semantic pars- ing corpora include sentences labeled with the sys- tem response or the target formal representation, and omit derivation information. CCG induction from such data relies on latent-variable techniques and re- quires careful initialization (e.g., Zettlemoyer and Collins, 2005, 2007). Such feature initialization does not directly transfer to a neural network archi- tecture with dense embeddings, and the use of hid- den layers further complicates learning by adding a large number of latent variables. We focus on data that includes sentence-representation pairs, and learn from a previously induced log-linear CKY parser. This drastically simplifies learning, and can be viewed as bootstrapping a fast parser from a slow one. While this dramatically …

Content truncated to 500 words