Dependency parsing with latent refinements of part-of-speech tags

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 963–967, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Dependency parsing with latent refinements of part-of-speech tags Thomas M¨uller†, Richard Farkas§, Alex Judea‡, Helmut Schmid†, and Hinrich Sch¨utze† †Center for Information and Language Processing, University of Munich, Germany §Department of Informatics, University of Szeged, Hungary ‡Heidelberg Institute for Theoretical Studies, Heidelberg, Germany muellets@cis.lmu.de Abstract In this paper we propose a method to increase dependency parser performance without using additional labeled or unla- beled data by refining the layer of pre- dicted part-of-speech (POS) tags. We per- form experiments on English and Ger- man and show significant improvements for both languages. The refinement is based on generative split-merge training for Hidden Markov models (HMMs). 1 Introduction Probabilistic Context-free Grammars with latent annotations (PCFG-LA) have been shown (Petrov et al., 2006) to yield phrase structure parsers with state-of-the-art accuracy. While Hidden Markov Models with latent annotations (HMM- LA) (Huang et al., 2009), stay somewhat behind the performance of state-of-the-art discriminative taggers (Eidelman et al., 2010). In this paper we address the question of whether the resulting la- tent POS tags are linguistically meaningful and useful for upstream tasks such as syntactic pars- ing. We find that this is indeed the case, lead- ing to a procedure that significantly increases the performance of dependency parsers. The proce- dure is attractive because the refinement of pre- dicted part-of-speech sequences using a coarse-to- fine strategy (Petrov and Klein, 2007) is fast and efficient. More precisely, we show that incorpo- rating the induced POS into a state-of-the-art de- pendency parser (Bohnet, 2010) gives increases in Labeled Attachment Score (LAS): from 90.34 to 90.57 for English and from 87.92 to 88.24 (resp. 88.35 to 88.51) for German without using (resp. with using) morphological features. 2 Related Work Petrov et al. (2006) introduce generative split- merge training for PCFGs and provide a fully au- tomatic method for training state-of-the-art phrase structure parsers. They argue that the resulting la- tent annotations are linguistically meaningful. Sun et al. (2008) induce latent sub-states into CRFs and show that noun phrase (NP) recognition can be im- proved, especially if no part-of-speech features are available. Huang et al. (2009) apply split-merge training to create HMMs with latent annotations (HMM-LA) for Chinese POS tagging. They re- port that the method outperforms standard gener- ative bigram and trigram tagging, but do not com- pare to discriminative methods. Eidelman et al. (2010) show that a bidirectional variant of latent HMMs with incorporation of prosodic information can yield state-of-the-art results in POS tagging of conversational speech. 3 Split-Merge Training for HMMs Split-merge training for HMMs (Huang et al., 2009) iteratively splits every tag into two subtags. Word emission and tag transition probabilities of subtags are then initialized close to the values of the parent tags but with some randomness to break symmetry. Using expectation–maximization (EM) training the parameters can then be set to a local maximum of the training data likelihood. …

Content truncated to 500 words