Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3912–3926 May 2-6, 2023 ©2023 Association for Computational Linguistics Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models Abteen Ebrahimi♢ Arya D. McCarthy∇ Arturo Oncevay♡ Luis Chiruzzo△ John E. OrtegaΩ Gustavo A. Giménez-Lugo♣ Rolando Coto-Solanoϕ Katharina Kann♢ ♢University of Colorado Boulder ∇Johns Hopkins University ♡University of Edinburgh △Universidad de la República, Uruguay ΩNortheastern University ♣Universidade Tecnológica Federal do Paraná ϕDartmouth College Abstract Large multilingual models have inspired a new class of word alignment methods, which work well for the model’s pretraining languages. However, the languages most in need of automatic alignment are low-resource and, thus, not typically included in the pretraining data. In this work, we ask: How do modern aligners perform on unseen languages, and are they better than traditional methods? We contribute gold-standard alignments for Bribri– Spanish, Guarani–Spanish, Quechua–Spanish, and Shipibo-Konibo–Spanish. With these, we evaluate state-of-the-art aligners with and without model adaptation to the target lan- guage. Finally, we also evaluate the resulting alignments extrinsically through two down- stream tasks: named entity recognition and part-of-speech tagging. We find that although transformer-based methods generally outper- form traditional models, the two classes of approach remain competitive with each other. 1 Introduction Word alignment is a valuable tool for extending the coverage of natural language processing (NLP) applications to low-resource languages through, e.g., statistical machine translation (SMT; Koehn and Knowles, 2017; Duh et al., 2020) or anno- tation projection (Yarowsky et al., 2001; Smith and Smith, 2004; Nicolai et al., 2020; Eskan- der et al., 2020). The traditional approach for generating alignments has been with statistical methods such as Giza++ (Och and Ney, 2003) and FastAlign (Dyer et al., 2013), which provide strong alignment quality while remaining quick and lightweight to run. Recently, new methods have been proposed which extract alignments from massive pretrained multilingual models, and out- perform these longstanding methods (Dou and Neubig, 2021). Our code and data can be found at https://github. com/abteen/alignment. CIA×××× ××× × × × ×× nisqam película nisqata uraykachirqa hinaspam Naciones Unidasman paqarintanta aparurqa . La CIA descargó la película y la llevó a las Naciones Unidas al día siguiente . × × × Figure 1: A word alignment between Quechua and Spanish (shaded), as well as mBERT+TLM’s predicted alignment (marked by ×’s). FastAlign and Giza++ can- not take advantage of surface features of proper names and borrowings. We evaluate alignments intrinsically via AER and extrinsically with POS-tagging and NER models learned on annotations projected across align- ments from Spanish. However, results on other NLP tasks, such as part-of-speech (POS) tagging and named-entity recognition (NER), have shown that, while pre- trained models generally work well out-of-the-box for high-resource languages, performance is far lower for low-resource ones, particularly those which are unseen during pretraining (Pires et al., 2019; Wu and Dredze, 2020; Muller et al., 2021; Lee et al., 2022). Models can be adapted (Guru- rangan et al., 2020; Chau et al., 2020) to …

Content truncated to 500 words