Ten Pairs to Tag – Multilingual POS Tagging via Coarse Mapping between Embeddings

Proceedings of NAACL-HLT 2016, pages 1307–1317, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Ten Pairs to Tag – Multilingual POS Tagging via Coarse Mapping between Embeddings Yuan Zhang, David Gaddy, Regina Barzilay, and Tommi Jaakkola Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology {yuanzh, dgaddy, regina, tommi}@csail.mit.edu Abstract In the absence of annotations in the target lan- guage, multilingual models typically draw on extensive parallel resources. In this paper, we demonstrate that accurate multilingual part- of-speech (POS) tagging can be done with just a few (e.g., ten) word translation pairs. We use the translation pairs to establish a coarse linear isometric (orthonormal) mapping be- tween monolingual embeddings. This en- ables the supervised source model expressed in terms of embeddings to be used directly on the target language. We further refine the model in an unsupervised manner by initial- izing and regularizing it to be close to the di- rect transfer model. Averaged across six lan- guages, our model yields a 37.5% absolute improvement over the monolingual prototype- driven method (Haghighi and Klein, 2006) when using a comparable amount of super- vision. Moreover, to highlight key linguistic characteristics of the generated tags, we use them to predict typological properties of lan- guages, obtaining a 50% error reduction rela- tive to the prototype model.1 1 Introduction After two decades of study, the best performing mul- tilingual methods can in some cases approach their supervised monolingual analogues. To reach this level of performance, however, multilingual meth- ods typically make use of significant parallel re- sources such as parallel translations or bilingual dic- 1Our code and data are available at https://github. com/yuanzh/transfer_pos. tionaries. These resources act as substitutes for ex- plicit annotations available in the target language for supervised methods. It is less clear what can be done without extensive parallel resources. Indeed, the motivation for our paper comes from trying to understand how little parallel data is necessary for effective multilingual transfer. In this paper, we demonstrate that only ten word translation pairs suffice for effective multilingual transfer of part-of-speech (POS) tagging. To achieve this we make use of and integrate two sources of statistical signal. First, we enable transfer of infor- mation from the source to target languages by es- tablishing a coarse mapping between word embed- dings in two languages on the basis of the few avail- able translation pairs. The mapping is useful be- cause of significant structural similarity of embed- ding spaces across languages. Second, we lever- age the potential of unsupervised monolingual mod- els to capture language-specific syntactic properties. The two sources of signals are largely complemen- tary. Embeddings provide a coarse alignment be- tween languages while unsupervised methods fine tune the correspondences in service of the task at hand. While unsupervised methods are fragile and challenging to estimate in general, they can be help- ful if initialized and regularized properly, which is our focus. In order to transfer annotations, we align mono- lingual embeddings between languages. However, a full fine-grained alignment …

Content truncated to 500 words