Experiments with Generative Models for Dependency Tree Linearization

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1978–1983, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Experiments with Generative Models for Dependency Tree Linearization Richard Futrell and Edward Gibson Department of Brain and Cognitive Sciences Massachusetts Institute of Technology {futrell, egibson}@mit.edu Abstract We present experiments with generative models for linearization of unordered la- beled syntactic dependency trees (Belz et al., 2011; Rajkumar and White, 2014). Our linearization models are derived from generative models for dependency struc- ture (Eisner, 1996). We present a series of generative dependency models designed to capture successively more information about ordering constraints among sister dependents. We give a dynamic program- ming algorithm for computing the condi- tional probability of word orders given tree structures under these models. The models are tested on corpora of 11 languages us- ing test-set likelihood, and human ratings for generated forms are collected for En- glish. Our models benefit from represent- ing local order constraints among sisters and from backing off to less sparse distri- butions, including distributions not condi- tioned on the head. 1 Introduction We explore generative models for producing lin- earizations of unordered labeled syntactic depen- dency trees. This specific task has attracted at- tention in recent years (Filippova and Strube, 2009; He et al., 2009; Belz et al., 2011; Bohnet et al., 2012; Zhang, 2013) because it forms a useful part of a natural language generation pipeline, especially in machine translation (Chang and Toutanova, 2007) and summarization (Barzi- lay and McKeown, 2005). Closely related tasks are generation of sentences given CCG parses (White and Rajkumar, 2012), bags of words (Liu et al., 2015), and semantic graphs (Braune et al., 2014). Here we focus narrowly on testing probabilistic generative models for dependency tree lineariza- tion. In contrast, the approach in most previ- ous work is to apply a variety of scoring func- tions to trees and linearizations and search for an optimally-scoring tree among some set. The prob- abilistic linearization models we investigate are derived from generative models for dependency trees (Eisner, 1996), as most commonly used in unsupervised grammar induction (Klein and Man- ning, 2004; Gelling et al., 2012). Generative de- pendency models have typically been evaluated in a parsing task (Eisner, 1997). Here, we are interested in the inverse task: inferring a distri- bution over linear orders given unordered depen- dency trees. This is the first work to consider generative de- pendency models from the perspective of word ordering. The results can potentially shed light on how ordering constraints are best represented in such models. In addition, the use of proba- bilistic models means that we can easily define well-motivated normalized probability distribu- tions over orders of dependency trees. These dis- tributions are useful for answering scientific ques- tions about crosslinguistic word order in quan- titative linguistics, where obtaining robust esti- mates has proven challenging due to data sparsity (Futrell et al., 2015). The remainder of the work is organized as fol- lows. In Section 2 we present …

Content truncated to 500 words