An Effective Neural Network Model for Graph-based Dependency Parsing

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 313–322, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics An Effective Neural Network Model for Graph-based Dependency Parsing Wenzhe Pei Tao Ge Baobao Chang∗ Key Laboratory of Computational Linguistics, Ministry of Education, School of Electronics Engineering and Computer Science, Peking University, No.5 Yiheyuan Road, Haidian District, Beijing, 100871, China Collaborative Innovation Center for Language Ability, Xuzhou, 221009, China. {peiwenzhe,getao,chbb}@pku.edu.cn Abstract Most existing graph-based parsing models rely on millions of hand-crafted features, which limits their generalization ability and slows down the parsing speed. In this paper, we propose a general and effective Neural Network model for graph-based dependency parsing. Our model can auto- matically learn high-order feature combi- nations using only atomic features by ex- ploiting a novel activation function tanh- cube. Moreover, we propose a simple yet effective way to utilize phrase-level infor- mation that is expensive to use in conven- tional graph-based parsers. Experiments on the English Penn Treebank show that parsers based on our model perform better than conventional graph-based parsers. 1 Introduction Dependency parsing is essential for computers to understand natural languages, whose performance may have a direct effect on many NLP applica- tion. Due to its importance, dependency parsing, has been studied for tens of years. Among a vari- ety of dependency parsing approaches (McDonald et al., 2005; McDonald and Pereira, 2006; Car- reras, 2007; Koo and Collins, 2010; Zhang and Nivre, 2011), graph-based models seem to be one of the most successful solutions to the challenge due to its ability of scoring the parsing decisions on whole-tree basis. Typical graph-based models factor the dependency tree into subgraphs, rang- ing from the smallest edge (first-order) to a con- trollable bigger subgraph consisting of more than one single edge (second-order and third order), and score the whole tree by summing scores of the subgraphs. In these models, subgraphs are usually represented as a high-dimensional feature vectors ∗Corresponding author which are fed into a linear model to learn the fea- ture weight for scoring the subgraphs. In spite of their advantages, conventional graph- based models rely heavily on an enormous num- ber of hand-crafted features, which brings about serious problems. First, a mass of features could put the models in the risk of overfitting and slow down the parsing speed, especially in the high- order models where combinational features cap- turing interactions between head, modifier, sib- lings and (or) grandparent could easily explode the feature space. In addition, feature design re- quires domain expertise, which means useful fea- tures are likely to be neglected due to a lack of domain knowledge. As a matter of fact, these two problems exist in most graph-based models, which have stuck the development of dependency parsing for a few years. To ease the problem of feature engineering, we propose a general and effective Neural Network model for graph-based dependency parsing in this paper. The main advantages of …

Content truncated to 500 words