Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech Tagging

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1577 - 1589 December 7-11, 2022 ©2022 Association for Computational Linguistics Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech Tagging Ayyoob Imani *1, Silvia Severini*1, Masoud Jalili Sabet1, François Yvon2, Hinrich Schütze1 1Center for Information and Language Processing (CIS), LMU Munich, Germany 2Université Paris-Saclay, CNRS, LISN, France {ayyoob, silvia, masoud}@cis.lmu.de, francois.yvon@limsi.fr Abstract Part-of-Speech (POS) tagging is an important component of the NLP pipeline, but many low- resource languages lack labeled data for train- ing. An established method for training a POS tagger in such a scenario is to create a labeled training set by transferring from high-resource languages. In this paper, we propose a novel method for transferring labels from multiple high-resource source to low-resource target languages. We formalize POS tag projection as graph-based label propagation. Given trans- lations of a sentence in multiple languages, we create a graph with words as nodes and align- ment links as edges by aligning words for all language pairs. We then propagate node la- bels from source to target using a Graph Neu- ral Network augmented with transformer lay- ers. We show that our propagation creates training sets that allow us to train POS tag- gers for a diverse set of languages. When combined with enhanced contextualized em- beddings, our method achieves a new state-of- the-art for unsupervised POS tagging of low resource languages. 1 Introduction In many applications, Part-of-Speech (POS) tag- ging is an important part of the NLP pipeline. In recent years, high-accuracy POS taggers have been developed owing to advances in machine learning methods that combine pretraining on large unla- beled corpora and supervised fine-tuning on well- curated annotated datasets. This methodology only applies to a handful of high-resource (HR) lan- guages for which the necessary training data exists, leaving behind the majority of low-resource (LR) languages. When training resources are scarce, an established method for training POS taggers is to automatically generate the training data via cross- lingual transfer (Yarowsky and Ngai, 2001; Fossum and Abney, 2005; Agi´c et al., 2016; Eskander et al., *Equal contribution. actions NOUN Nw speak VERB louder ADV than PREP words NOUN taten NOUN sagen VERB mehr ADV als PREP worte NOUN eylemler ? sözlerden ? daha ? yüksek ? sesle ? konuşur ? دﻧﮐ ﯽﻣ ? تﺑﺣﺻ ? لﻣﻋ ? زا ? رﺗﺎﺳر ? مﻼﮐ ? Figure 1: The sentence “actions speak louder than words” in English and its translations to Persian, Ger- man, and Turkish, aligned at the word level. The POS tags for high-resource English and German are known. We use a GNN to exploit this graph structure and com- pute POS tags for low-resource Persian and Turkish. 2020). Typically, POS annotations are projected through alignment links from the HR source to the LR target of a word aligned parallel corpus. In this paper, we propose GLP (Graph Label Propagation), a novel method for transferring la- bels simultaneously from multiple high-resource source languages to multiple low-resource target languages. We …

Content truncated to 500 words