Using Universal Linguistic Knowledge to Guide Grammar Induction

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1234–1244, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Using Universal Linguistic Knowledge to Guide Grammar Induction Tahira Naseem, Harr Chen, Regina Barzilay Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology {tahira, harr, regina} @csail.mit.edu Mark Johnson Department of Computing Macquarie University mark.johnson@mq.edu.au Abstract We present an approach to grammar induc- tion that utilizes syntactic universals to im- prove dependency parsing across a range of languages. Our method uses a single set of manually-specified language-independent rules that identify syntactic dependencies be- tween pairs of syntactic categories that com- monly occur across languages. During infer- ence of the probabilistic model, we use pos- terior expectation constraints to require that a minimum proportion of the dependencies we infer be instances of these rules. We also auto- matically refine the syntactic categories given in our coarsely tagged input. Across six lan- guages our approach outperforms state-of-the- art unsupervised methods by a significant mar- gin.1 1 Introduction Despite surface differences, human languages ex- hibit striking similarities in many fundamental as- pects of syntactic structure. These structural corre- spondences, referred to as syntactic universals, have been extensively studied in linguistics (Baker, 2001; Carnie, 2002; White, 2003; Newmeyer, 2005) and underlie many approaches in multilingual parsing. In fact, much recent work has demonstrated that learning cross-lingual correspondences from cor- pus data greatly reduces the ambiguity inherent in syntactic analysis (Kuhn, 2004; Burkett and Klein, 2008; Cohen and Smith, 2009a; Snyder et al., 2009; Berg-Kirkpatrick and Klein, 2010). 1The source code for the work presented in this paper is available at http://groups.csail.mit.edu/rbg/code/dependency/ Root → Auxiliary Noun → Adjective Root → Verb Noun → Article Verb → Noun Noun → Noun Verb → Pronoun Noun → Numeral Verb → Adverb Preposition → Noun Verb → Verb Adjective → Adverb Auxiliary → Verb Table 1: The manually-specified universal dependency rules used in our experiments. These rules specify head- dependent relationships between coarse (i.e., unsplit) syntactic categories. An explanation of the ruleset is pro- vided in Section 5. In this paper, we present an alternative gram- mar induction approach that exploits these struc- tural correspondences by declaratively encoding a small set of universal dependency rules. As input to the model, we assume a corpus annotated with coarse syntactic categories (i.e., high-level part-of- speech tags) and a set of universal rules defined over these categories, such as those in Table 1. These rules incorporate the definitional properties of syn- tactic categories in terms of their interdependencies and thus are universal across languages. They can potentially help disambiguate structural ambiguities that are difficult to learn from data alone — for example, our rules prefer analyses in which verbs are dependents of auxiliaries, even though analyz- ing auxiliaries as dependents of verbs is also consis- tent with the data. Leveraging these universal rules has the potential to improve parsing performance for a large number of human languages; this is par- ticularly relevant to the processing …

Content truncated to 500 words