A Diverse Dirichlet Process Ensemble for Unsupervised Induction of Syntactic Categories

Proceedings of COLING 2012: Technical Papers, pages 2307–2324, COLING 2012, Mumbai, December 2012. A Diverse Dirichlet Process Ensemble for Unsupervised Induction of Syntactic Categories Roi Reichart1 Gal Elidan2 Ari Rappoport3 (1) The Computer Laboratory, University of Cambridge, UK (2) Department of Statistics, The Hebrew University (3) Institute of computer science, The Hebrew University ❘♦✐✳❘❡✐❝❤❛rt❅❝❧✳❝❛♠✳❛❝✳✉❦✱ ❣❛❧❡❧❅❤✉❥✐✳❛❝✳✐❧✱ ❛r✐r❅❝s✳❤✉❥✐✳❛❝✳✐❧ ABSTRACT We address the problem of unsupervised tagging of phrase structure trees with phrase categories (parse tree nonterminals). Motivated by the inability of a range of direct clustering approaches to improve over the current leading algorithm, we propose a mixture of experts approach. In particular, we tackle the difficult challenge of producing a diverse collection of useful tagging experts, which can then be aggregated into a final high-quality tagging. To do so, we use the particular properties of the Dirichlet Process mixture model. We evaluate on English, German and Chinese corpora and demonstrate both a substantial and consistent improvement in overall performance over previous work, as well as empirical justification of our algorithmic choices. KEYWORDS: Unsupervised parsing, Grammar induction, Non terminals, Dirichlet Process, Ensemble learning. 2307 1 Introduction Grammar induction is the task of learning grammatical structure from plain text without human supervision. The task is valuable for the understanding of human language acquisition and its output can potentially be used by NLP applications, avoiding the costly and error prone creation of manually annotated corpora. The task has been widely explored (Klein, 2005) and its importance has increased due to the recent availability of huge corpora. The induced grammar can be represented in various ways. Most work (e.g., (Klein and Manning, 2004; Smith and Eisner, 2006; Seginer, 2007; Headden et al., 2009)) annotate text sentences using an unlabeled hierarchical phrase or a dependency structure, and thus represent the induced grammar through its behaviour in a parsing task. An important task in theory and practice that we consider in this work is how to enrich phrase structures with syntactic categories. The two grammars that have been widely explored by the NLP community in the last two decades, phrase structure grammars and dependency grammars, allow the induced structure to be either labeled or not and use the labeling to describe substantially different syntactic functions. In this paper we focus on the former formalism and induce parse tree nonterminals (e.g., ‘NP’, ‘VP’, ‘PP’) in an unsupervised manner. We keep the discussion of dependency parsing for future research. Many linguistic theories posit a hierarchical labeled constituent (or constructional) structure, arguing that it has a measurable psychological reality (e.g., (Goldberg, 2006)). Practically, most of the syntactic annotation of corpora used by the NLP community comes in the form of labeled structures. Indeed, modern supervised syntactic parsers aim at learning labeled structures. Moreover, phrase categories are often used in a variety of NLP tasks, such as SRL (Gildea and Jurafsky, 2002; punyakanok et al., 2008), alignment in syntax-based machine translation (Zhang and Gildea, 2004), information extraction (Miyao et al., 2008), etc. Phrase categories can be induced either jointly with the …

Content truncated to 500 words