Transition-based Neural Constituent Parsing

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1169–1179, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Transition-based Neural Constituent Parsing Taro Watanabe∗and Eiichiro Sumita National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289 JAPAN tarow@google.com, eiichiro.sumita@nict.go.jp Abstract Constituent parsing is typically modeled by a chart-based algorithm under prob- abilistic context-free grammars or by a transition-based algorithm with rich fea- tures. Previous models rely heavily on richer syntactic information through lex- icalizing rules, splitting categories, or memorizing long histories. However en- riched models incur numerous parameters and sparsity issues, and are insufficient for capturing various syntactic phenomena. We propose a neural network structure that explicitly models the unbounded history of actions performed on the stack and queue employed in transition-based parsing, in addition to the representations of partially parsed tree structure. Our transition-based neural constituent parsing achieves perfor- mance comparable to the state-of-the-art parsers, demonstrating F1 score of 90.68% for English and 84.33% for Chinese, with- out reranking, feature templates or addi- tional data to train model parameters. 1 Introduction A popular parsing algorithm is a cubic time chart- based dynamic programming algorithm that uses probabilistic context-free grammars (PCFGs). However, PCFGs learned from treebanks are too coarse to represent the syntactic structures of texts. To address this problem, various contexts are in- corporated into the grammars through lexicaliza- tion (Collins, 2003; Charniak, 2000) or cate- gory splitting either manually (Klein and Man- ning, 2003) or automatically (Matsuzaki et al., 2005; Petrov et al., 2006). Recently a rich feature set was introduced to capture the lexical contexts ∗The first author is now affiliated with Google, Japan. in each span without extra annotations in gram- mars (Hall et al., 2014). Alternatively, transition-based algorithms run in linear time by taking a series of shift-reduce ac- tions with richer lexicalized features considering histories; however, the accuracies did not match with the state-of-the-art methods until recently (Sagae and Lavie, 2005; Zhang and Clark, 2009). Zhu et al. (2013) show that the use of better transi- tion actions considering unaries and a set of non- local features can compete with the accuracies of chart-based parsing. The features employed in a transition-based algorithm usually require part of speech (POS) annotation in the input, but the de- layed feature technique allows joint POS inference (Wang and Xue, 2014). In both frameworks, the richer models require that more parameters be estimated during train- ing which can easily result in the data sparseness problems. Furthermore, the enriched models are still insufficient to capture various syntactic rela- tions in texts due to the limited contexts repre- sented in latent annotations or non-local features. Recently Socher et al. (2013) introduced composi- tional vector grammar (CVG) to address the above limitations. However, they employ reranking over a forest generated by a baseline parser for efficient search, because CVG is built on cubic time chart- based parsing. In this paper, we propose a …

Content truncated to 500 words