Cross-Lingual Part-of-Speech Tagging through Ambiguous Learning

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1779–1785, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Cross-Lingual Part-of-Speech Tagging through Ambiguous Learning Guillaume Wisniewski Nicolas Pécheux Souhir Gahbiche-Braham François Yvon Université Paris Sud LIMSI-CNRS 91 403 ORSAY CEDEX, France {wisniews,pecheux,souhir,yvon}@limsi.fr Abstract When Part-of-Speech annotated data is scarce, e.g. for under-resourced lan- guages, one can turn to cross-lingual trans- fer and crawled dictionaries to collect par- tially supervised data. We cast this prob- lem in the framework of ambiguous learn- ing and show how to learn an accurate history-based model. Experiments on ten languages show significant improvements over prior state of the art performance. 1 Introduction In the past two decades, supervised Machine Learning techniques have established new perfor- mance standards for many NLP tasks. Their suc- cess however crucially depends on the availability of annotated in-domain data, a not so common sit- uation. This means that for many application do- mains and/or less-resourced languages, alternative ML techniques need to be designed to accommo- date unannotated or partially annotated data. Several attempts have recently been made to mitigate the lack of annotated corpora using par- allel data pairing a (source) text in a resource-rich language with its counterpart in a less-resourced language. By transferring labels from the source to the target, it becomes possible to obtain noisy, yet useful, annotations that can be used to train a model for the target language in a weakly super- vised manner. This research trend was initiated by Yarowsky et al. (2001), who consider the trans- fer of POS and other syntactic information, and further developed in (Hwa et al., 2005; Ganchev et al., 2009) for syntactic dependencies, in (Padó and Lapata, 2009; Kozhevnikov and Titov, 2013; van der Plas et al., 2014) for semantic role la- beling and in (Kim et al., 2012) for named-entity recognition, to name a few. Assuming that labels can actually be projected across languages, these techniques face the issue of extending standard supervised techniques with partial and/or uncertain labels in the presence of alignment noise. In comparison to the early ap- proach of Yarowsky et al. (2001) in which POS are directly transferred, subject to heuristic fil- tering rules, recent works consider the integra- tion of softer constraints using expectation regu- larization techniques (Wang and Manning, 2014), the combination of alignment-based POS transfer with additional information sources such as dic- tionaries (Li et al., 2012; Täckström et al., 2013) (Section 2), or even the simultaneous use of both techniques (Ganchev and Das, 2013). In this paper, we reproduce the weakly super- vised setting of Täckström et al. (2013). By re- casting this setting in the framework of ambiguous learning (Bordes et al., 2010; Cour et al., 2011) (Section 3), we propose an alternative learning methodology and show that it improves the state of the art performance on a large array of languages (Section 4). Our analysis of the remaining errors suggests that in cross-lingual settings, improve- ments of error rates …

Content truncated to 500 words