A Grounded Unsupervised Universal Part-of-Speech Tagger for Low-Resource Languages

Proceedings of NAACL-HLT 2019, pages 2428–2439 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2428 A Grounded Unsupervised Universal Part-of-Speech Tagger for Low-Resource Languages Ronald Cardenas♣ Ying Lin♠ Heng Ji♠ Jonathan May♥ ♣ Institute of Formal and Applied Linguistics, Charles University in Prague ♠ Computer Science Department, Rensselaer Polytechnic Institute ♥ Information Sciences Institute, University of Southern California ronald.cardenas@matfyz.cz liny9@rpi.edu jih@rpi.edu jonmay@isi.edu Abstract Unsupervised part of speech (POS) tagging is often framed as a clustering problem, but prac- tical taggers need to ground their clusters as well. Grounding generally requires reference labeled data, a luxury a low-resource language might not have. In this work, we describe an approach for low-resource unsupervised POS tagging that yields fully grounded output and requires no labeled training data. We find the classic method of Brown et al. (1992) clusters well in our use case and employ a decipherment-based approach to grounding. This approach presumes a sequence of cluster IDs is a ‘ciphertext’ and seeks a POS tag-to- cluster ID mapping that will reveal the POS sequence. We show intrinsically that, despite the difficulty of the task, we obtain reasonable performance across a variety of languages. We also show extrinsically that incorporating our POS tagger into a name tagger leads to state- of-the-art tagging performance in Sinhalese and Kinyarwanda, two languages with nearly no labeled POS data available. We further demonstrate our tagger’s utility by incorporat- ing it into a true ‘zero-resource’ variant of the MALOPA (Ammar et al., 2016) dependency parser model that removes the current reliance on multilingual resources and gold POS tags for new languages. Experiments show that in- cluding our tagger makes up much of the accu- racy lost when gold POS tags are unavailable. 1 Introduction While cellular, satellite, and hardware advances have ensured that sophisticated NLP technology can reach all corners of the earth, the language bar- rier upon reaching remote locales still remains. As an example, when international aid organizations respond to new disasters, they are often unable to deploy technology to understand local reports de- tailing specific events (Munro and Manning, 2012; Lewis et al., 2011). An inability to communicate Figure 1: Overview of our approach to grounded POS tagging. We use an unsupervised clustering method (Section 3.2) then reduce and ground the clusters us- ing a decipherment approach informed by POS tag se- quence data from many languages (Section 3.3). with partner governments or civilian populations in a timely manner leads to preventable casualties. The lack of adequate labeled training data has been the major obstacle to expanding NLP’s out- reach more multilingually. Developments in un- supervised techniques that require only monolin- gual corpora (Lample et al., 2018a; Artetxe et al., 2018) and the ability to leverage labeled resources in other languages have been proposed to address this issue (Das and Petrov, 2011; Duong et al., 2014; Ammar et al., 2016). Unfortunately, these 2429 methods either do not work in practice on true low- resource cases or unrealistically assume the avail- …

Content truncated to 500 words