Transition-based Dependency Parsing Using Two Heterogeneous Gated Recursive Neural Networks

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1879–1889, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Transition-based Dependency Parsing Using Two Heterogeneous Gated Recursive Neural Networks Xinchi Chen, Yaqian Zhou, Chenxi Zhu, Xipeng Qiu, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {xinchichen13,zhouyaqian,czhu13,xpqiu,xjhuang}@fudan.edu.cn Abstract Recently, neural network based depen- dency parsing has attracted much interest, which can effectively alleviate the prob- lems of data sparsity and feature engineer- ing by using the dense features. How- ever, it is still a challenge problem to sufficiently model the complicated syn- tactic and semantic compositions of the dense features in neural network based methods. In this paper, we propose two heterogeneous gated recursive neu- ral networks: tree structured gated re- cursive neural network (Tree-GRNN) and directed acyclic graph structured gated recursive neural network (DAG-GRNN). Then we integrate them to automati- cally learn the compositions of the dense features for transition-based dependency parsing. Specifically, Tree-GRNN mod- els the feature combinations for the trees in stack, which already have partial depen- dency structures. DAG-GRNN models the feature combinations of the nodes whose dependency relations have not been built yet. Experiment results on two prevalent benchmark datasets (PTB3 and CTB5) show the effectiveness of our proposed model. 1 Introduction Transition-based dependency parsing is a core task in natural language processing, which has been studied with considerable efforts in the NLP com- munity. The traditional discriminative dependency parsing methods have achieved great success (Koo et al., 2008; He et al., 2013; Bohnet, 2010; Huang and Sagae, 2010; Zhang and Nivre, 2011; Mar- tins et al., 2009; McDonald et al., 2005; Nivre et al., 2006; K¨ubler et al., 2009; Goldberg and Nivre, (a) Standard RNN (b) Tree-GRNN (c) DAG-GRNN Figure 1: Sketch of three recursive neural net- works (RNN). (a) is the standard RNN for con- stituent tree; (b) is Tree-GRNN for dependency tree, in which the dashed arcs indicate the depen- dency relations between the nodes; (c) is DAG- GRNN for the nodes without given topological structure. 2013; Choi and McCallum, 2013; Ballesteros and Bohnet, 2014). However, these methods are based on discrete features and suffer from the problems of data sparsity and feature engineering (Chen and Manning, 2014). Recently, distributed representations have been widely used in a variety of natural language pro- cessing (NLP) tasks (Collobert et al., 2011; De- vlin et al., 2014; Socher et al., 2013; Turian et al., 2010; Mikolov et al., 2013b; Bengio et al., 2003). Specific to the transition-based parsing, the neu- ral network based methods have also been increas- ingly focused on due to their ability to minimize the efforts in feature engineering and the boosted performance (Le and Zuidema, 2014; Stenetorp, 2013; Bansal et al., 2014; Chen and Manning, 2014; Zhu et al., 2015). However, most of the existing neural network based methods still need some efforts in feature engineering. For example, most methods often se- lect …

Content truncated to 500 words