Easy-First Dependency Parsing with Hierarchical Tree LSTMs

Easy-First Dependency Parsing with Hierarchical Tree LSTMs Eliyahu Kiperwasser Computer Science Department Bar-Ilan University Ramat-Gan, Israel elikip@gmail.com Yoav Goldberg Computer Science Department Bar-Ilan University Ramat-Gan, Israel yoav.goldberg@gmail.com Abstract We suggest a compositional vector represen- tation of parse trees that relies on a recursive combination of recurrent-neural network en- coders. To demonstrate its effectiveness, we use the representation as the backbone of a greedy, bottom-up dependency parser, achiev- ing very strong accuracies for English and Chinese, without relying on external word embeddings. The parser’s implementation is available for download at the first author’s webpage. 1 Introduction Dependency-based syntactic representations of sen- tences are central to many language processing tasks (Kübler et al., 2009). Dependency parse-trees en- code not only the syntactic structure of a sentence but also many aspects of its semantics. A recent trend in NLP is concerned with encod- ing sentences as vectors (“sentence embeddings”), which can then be used for further prediction tasks. Recurrent neural networks (RNNs) (Elman, 1990), and in particular methods based on the LSTM archi- tecture (Hochreiter and Schmidhuber, 1997), work very well for modeling sequences, and constantly obtain state-of-the-art results on both language- modeling and prediction tasks (see, e.g. (Mikolov et al., 2010)). Several works attempt to extend recurrent neu- ral networks to work on trees (see Section 8 for a brief overview), giving rise to the so-called recursive neural networks (Goller and Kuchler, 1996; Socher et al., 2010). However, recursive neural networks do not cope well with trees with arbitrary branch- ing factors – most work require the encoded trees to be binary-branching, or have a fixed maximum arity. Other attempts allow arbitrary branching factors, at the expense of ignoring the order of the modifiers. In contrast, we propose a tree-encoding that nat- urally supports trees with arbitrary branching fac- tors, making it particularly appealing for depen- dency trees. Our tree encoder uses recurrent neural networks as a building block: we model the left and right sequences of modifiers using RNNs, which are composed in a recursive manner to form a tree (Sec- tion 3). We use our tree representation for encoding the partially-built parse trees in a greedy, bottom-up dependency parser which is based on the easy-first transition-system of Goldberg and Elhadad (2010). Using the Hierarchical Tree LSTM representa- tion, and without using any external embeddings, our parser achieves parsing accuracies of 92.6 UAS and 90.2 LAS on the PTB (Stanford dependencies) and 86.1 UAS and 84.4 LAS on the Chinese tree- bank, while relying on greedy decoding. To the best of our knowledge, this is the first work to demonstrate competitive parsing accuracies for full-scale parsing while relying solely on recursive, compositional tree representations, and without us- ing a reranking framework. We discuss related work in Section 8. While the parsing experiments demonstrate the suitability of our representation for capturing the structural elements in the parse tree that are useful for predicting parsing decisions, we are interested in exploring the use of the RNN-based compositional vector representation of parse trees also …

Content truncated to 500 words