Neural Greedy Constituent Parsing with Dynamic Oracles

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 172–182, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Neural Greedy Constituent Parsing with Dynamic Oracles Maximin Coavoux1,2 and Benoˆıt Crabb´e1,2,3 1Univ. Paris Diderot, Sorbonne Paris Cit´e 2Alpage, Inria 3Institut Universitaire de France maximin.coavoux@inria.fr benoit.crabbe@linguist.univ-paris-diderot.fr Abstract Dynamic oracle training has shown sub- stantial improvements for dependency parsing in various settings, but has not been explored for constituent parsing. The present article introduces a dynamic ora- cle for transition-based constituent pars- ing. Experiments on the 9 languages of the SPMRL dataset show that a neu- ral greedy parser with morphological fea- tures, trained with a dynamic oracle, leads to accuracies comparable with the best non-reranking and non-ensemble parsers. 1 Introduction Constituent parsing often relies on search methods such as dynamic programming or beam search, be- cause the search space of all possible predictions is prohibitively large. In this article, we present a greedy parsing model. Our main contribution is the design of a dynamic oracle for transition- based constituent parsing. In NLP, dynamic or- acles were first proposed to improve greedy de- pendency parsing training without involving addi- tional computational costs at test time (Goldberg and Nivre, 2012; Goldberg and Nivre, 2013). The training of a transition-based parser in- volves an oracle, that is a function mapping a con- figuration to the best transition. Transition-based parsers usually rely on a static oracle, only well- defined for gold configurations, which transforms trees into sequences of gold actions. Training against a static oracle restricts the exploration of the search space to the gold sequence of actions. At test time, due to error propagation, the parser will be in a very different situation than at train- ing time. It will have to infer good actions from noisy configurations. To alleviate error propaga- tion, a solution is to train the parser to predict the best action given any configuration, by allowing it to explore a greater part of the search space at train time. Dynamic oracles are non-deterministic oracles well-defined for any configuration. They give the best possible transitions for any config- uration. Although dynamic oracles are widely used in dependency parsing and available for most standard transition systems (Goldberg and Nivre, 2013; Goldberg et al., 2014; G´omez-Rodr´ıguez et al., 2014; Straka et al., 2015), no dynamic oracle parsing model has yet been proposed for phrase structure grammars. The model we present aims at parsing mor- phologically rich languages (MRL). Recent re- search has shown that morphological features are very important for MRL parsing (Bj¨orkelund et al., 2013; Crabb´e, 2015). However, traditional linear models (such as the structured perceptron) need to define rather complex feature templates to capture interactions between features. Addi- tional morphological features complicate this task (Crabb´e, 2015). Instead, we propose to rely on a neural network weighting function which uses a non-linear hidden layer to automatically capture interactions between variables, and embeds mor- phological features in a vector space, as is usual for words and other symbols …

Content truncated to 500 words