Target Language Adaptation of Discriminative Transfer Parsers

Proceedings of NAACL-HLT 2013, pages 1061–1071, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Target Language Adaptation of Discriminative Transfer Parsers Oscar Täckström∗ SICS | Uppsala University Sweden oscar@sics.se Ryan McDonald Google New York ryanmcd@google.com Joakim Nivre∗ Uppsala University Sweden joakim.nivre@lingfil.uu.se Abstract We study multi-source transfer parsing for resource-poor target languages; specifically methods for target language adaptation of delexicalized discriminative graph-based de- pendency parsers. We first show how recent insights on selective parameter sharing, based on typological and language-family features, can be applied to a discriminative parser by carefully decomposing its model features. We then show how the parser can be relexicalized and adapted using unlabeled target language data and a learning method that can incorporate diverse knowledge sources through ambiguous labelings. In the latter scenario, we exploit two sources of knowledge: arc marginals de- rived from the base parser in a self-training algorithm, and arc predictions from multiple transfer parsers in an ensemble-training algo- rithm. Our final model outperforms the state of the art in multi-source transfer parsing on 15 out of 16 evaluated languages. 1 Introduction Many languages still lack access to core NLP tools, such as part-of-speech taggers and syntactic parsers. This is largely due to the reliance on fully supervised learning methods, which require large quantities of manually annotated training data. Recently, meth- ods for cross-lingual transfer have appeared as a promising avenue for overcoming this hurdle for both part-of-speech tagging (Yarowsky et al., 2001; Das and Petrov, 2011) and syntactic dependency parsing (Hwa et al., 2005; Zeman and Resnik, 2008; Ganchev et al., 2009; McDonald et al., 2011; Naseem et al., ∗Work primarily carried out while at Google, NY. 2012). While these methods do not yet compete with fully supervised approaches, they can drastically out- perform both unsupervised methods (Klein and Man- ning, 2004) and weakly supervised methods (Naseem et al., 2010; Berg-Kirkpatrick and Klein, 2010). A promising approach to cross-lingual transfer of syntactic dependency parsers is to use multiple source languages and to tie model parameters across related languages. This idea was first explored for weakly supervised learning (Cohen and Smith, 2009; Snyder et al., 2009; Berg-Kirkpatrick and Klein, 2010) and recently by Naseem et al. (2012) for multi- source cross-lingual transfer. In particular, Naseem et al. showed that by selectively sharing parameters based on typological features of each language, sub- stantial improvements can be achieved, compared to using a single set of parameters for all languages. However, these methods all employ generative mod- els with strong independence assumptions and weak feature representations, which upper bounds their ac- curacy far below that of feature-rich discriminative parsers (McDonald et al., 2005; Nivre, 2008). In this paper, we improve upon the state of the art in cross-lingual transfer of dependency parsers from multiple source languages by adapting feature-rich discriminatively trained parsers to a specific target language. First, in §4 we show how selective sharing of model parameters based on typological traits can be incorporated into a delexicalized discriminative graph-based parsing model. This requires a …

Content truncated to 500 words