A Search-Based Dynamic Reranking Model for Dependency Parsing

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1393–1402, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics A Search-Based Dynamic Reranking Model for Dependency Parsing Hao Zhou† Yue Zhang‡ Shujian Huang† Junsheng Zhou∗ Xin-Yu Dai† Jiajun Chen† †State Key Laboratory for Novel Software Technology, Nanjing University, China ‡Singapore University of Technology and Design, Singapore ∗ Nanjing Normal University, China zhouh@nlp.nju.edu.cn, yue zhang@sutd.edu.sg {huangshujian,daixinyu,chenjj}@nju.edu.cn, zhoujs@njnu.edu.cn Abstract We propose a novel reranking method to extend a deterministic neural dependency parser. Different to conventional k-best reranking, the proposed model integrates search and learning by utilizing a dynamic action revising process, using the rerank- ing model to guide modification for the base outputs and to rerank the candidates. The dynamic reranking model achieves an absolute 1.78% accuracy improvement over the deterministic baseline parser on PTB, which is the highest improvement by neural rerankers in the literature. 1 Introduction Neural network models have recently been ex- ploited for dependency parsing. Chen and Man- ning (2014) built a seminal model by replacing the SVM classifier at the transition-based Malt- Parser (Nivre et al., 2007) with a feed-forward neural network, achieving significantly higher ac- curacies and faster speed. As a local and greedy neural baseline, it does not outperform the best discrete-feature parsers, but nevertheless demon- strates strong potentials for neural network models in transition-based dependency parsing. Subsequent work aimed to improve the model of Chen and Manning (2014) in two main direc- tions. First, global optimization learning and beam search inference have been exploited to reduce er- ror propagation (Weiss et al., 2015; Zhou et al., 2015). Second, recurrent neural network models have been used to extend the range of neural fea- tures beyond a local window (Dyer et al., 2015; Ballesteros et al., 2015). These methods give ac- curacies that are competitive to the best results in the literature. John loves Mary (a) Base Tree John loves Mary (b) Tree 1 John loves Mary (c) Tree 2 1.0 S 1.0 S 0.4 S 0.5 L 0.9 L 0.3 L 1.0 S 0.5 L Revise 1 Tree 1 Tree 2 Base Tree Revise 2 1.0 S 1.0 S 0.3 L 1.0 S 0.5 R 1.0 S 1.0 S (d) 2-step action revising process for sentence “John loves Mary”. Numbers before actions are the probabilities for that action. Figure 1: Example action revising process. S, L, R stand for the SHIFT, LEFT, RIGHT actions, re- spectively (Section 2). Another direction to extend a baseline parser is reranking (Collins and Koo, 2000; Charniak and Johnson, 2005; Huang, 2008). Recently, neural network models have been used to con- stituent (Socher et al., 2013; Le et al., 2013) and dependency (Le and Zuidema, 2014; Zhu et al., 2015) parsing reranking. Compared with rerankers that rely on discrete manual features, neural network rerankers can potentially capture more global information over whole parse trees. Traditional rerankers are based on chart parsers, which can yield exact k-best lists and forests. For reranking, this is infeasible for …

Content truncated to 500 words