Parsing low-resource languages using Gibbs sampling for PCFGs with latent annotations

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 290–300, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Parsing low-resource languages using Gibbs sampling for PCFGs with latent annotations Liang Sun1 Jason Mielens2 1Department of Mechanical Engineering 2Department of Linguistics The University of Texas at Austin The University of Texas at Austin sally722@utexas.edu {jmielens,jbaldrid}@utexas.edu Jason Baldridge2 Abstract PCFGs with latent annotations have been shown to be a very effective model for phrase structure parsing. We present a Bayesian model and algorithms based on a Gibbs sam- pler for parsing with a grammar with latent an- notations. For PCFG-LA, we present an ad- ditional Gibbs sampler algorithm to learn an- notations from training data, which are parse trees with coarse (unannotated) symbols. We show that a Gibbs sampling technique is ca- pable of parsing sentences in a wide variety of languages and producing results that are on-par with or surpass previous approaches. Our results for Kinyarwanda and Malagasy in particular demonstrate that low-resource lan- guage parsing can benefit substantially from a Bayesian approach. 1 Introduction Despite great progress over the past two decades on parsing, relatively little work has considered the prob- lem of creating accurate parsers for low-resource lan- guages. Existing work in this area focuses primarily on approaches that use some form of cross-lingual boot- strapping to improve performance. For instance, Hwa et al. (2005) use a parallel Chinese/English corpus and an English dependency grammar to induce an anno- tated Chinese corpus in order to train a Chinese de- pendency grammar. Kuhn (2004b) also considers the benefits of using multiple languages to induce a mono- lingual grammar, making use of a measure for data re- liability in order to weight training data based on confi- dence of annotation. Bootstrapping approaches such as these achieve markedly improved results, but they are dependent on the existence of a parallel bilingual cor- pus. Very few such corpora are readily available, par- ticularly for low-resource languages, and creating such corpora obviously presents a challenge for many practi- cal applications. Kuhn (2004a) shows some of the diffi- culty in handling low-resource languages by examining various tasks using Q’anjob’al as an example. Another approach is that of Bender et al. (2002), who take a more linguistically-motivated approach by making use of linguistic universals to seed newly developed gram- mars. This substantially reduces the effort by making it unnecessary to learn the basic parameters of a lan- guage, but it lacks the robustness of grammars learned from data. Recent work on Probabilistic Context-Free Gram- mars with latent annotations (PCFG-LA) (Matsuzaki et al., 2005; Petrov et al., 2006) have shown them to be effective models for syntactic parsing, especially when less training material is available (Liang et al., 2009; Shindo et al., 2012). The coarse nonterminal symbols found in vanilla PCFGs are refined by latent variables; these latent annotations can model subtypes of gram- mar symbols that result in better grammars and enable better estimates of grammar productions. In this …

Content truncated to 500 words