Shift-Reduce CCG Parsing using Neural Network Models

Proceedings of NAACL-HLT 2016, pages 447–453, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Shift-Reduce CCG Parsing using Neural Network Models Bharat Ram Ambati and Tejaswini Deoskar and Mark Steedman Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh bharat.ambati@ed.ac.uk, {tdeoskar,steedman}@inf.ed.ac.uk Abstract We present a neural network based shift- reduce CCG parser, the first neural-network based parser for CCG. We also study the im- pact of neural network based tagging mod- els, and greedy versus beam-search parsing, by using a structured neural network model. Our greedy parser obtains a labeled F-score of 83.27%, the best reported result for greedy CCG parsing in the literature (an improve- ment of 2.5% over a perceptron based greedy parser) and is more than three times faster. With a beam, our structured neural network model gives a labeled F-score of 85.57% which is 0.6% better than the perceptron based counterpart. 1 Introduction Shift-reduce parsing is interesting for practical real- world applications like parsing the web, since pars- ing can be achieved in linear time. Although greedy parsers are fast, accuracies of these parsers are typically much lower than graph-based parsers. Conversely, beam-search parsers achieve accura- cies comparable to graph-based parsers (Zhang and Nivre, 2011) but are much slower than their greedy counterparts. Recently, Chen and Manning (2014) have showed that fast and accurate parsing can be achieved using neural network based parsers. Im- proving their work, Weiss et al. (2015) presented a structured neural network model which gave state- of-the-art results for English dependency parsing. There has been increasing interest in Combina- tory Categorial Grammar (CCG) (Steedman, 2000) parsing due to the simplicity of its interface between syntax and semantics. In addition to predicate- argument structure, CCG captures the unbounded dependencies found in grammatical constructions like relativization, coordination, etc. We present a neural network based shift-reduce CCG parser, the first neural network based parser for CCG. We first adapt Chen and Manning (2014)’s shift-reduce de- pendency parser for CCG parsing. We then develop a structured neural network model based on Weiss et al. (2015), in order to explore the impact of a beam- search on the parser. We also analyze the impact of neural network taggers (for both POS-tagging and CCG supertagging) as compared to maximum entropy taggers. Our greedy neural network parser achieves unlabeled and labeled F-scores of 89.78% and 83.27% respectively, an improvement of around 2.5% over a perceptron based greedy parser, and is more than three times faster. Due to its relevance for large-scale parsing, we make this parser available for public usage. By using a beam search, our structured neural network model gave even better results of 91.95% and 85.57% unlabeled and labeled F-scores respectively. To the best of our knowledge, ours is the first neural network based parser for CCG and also the first work on exploring neural network taggers for shift-reduce CCG parsing. 2 Related Work 2.1 CCG Parsers Due to the availability of English CCGbank (Hock- enmaier and Steedman, 2007), several wide- …

Content truncated to 500 words