Leveraging a Bilingual Dictionary to Learn Wolastoqey Word Representations

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1159–1166 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1159 Leveraging a Bilingual Dictionary to Learn Wolastoqey Word Representations Diego Bear, Paul Cook Faculty of Computer Science University of New Brunswick {diego.bear, paul.cook}@unb.ca Abstract Word embeddings (Mikolov et al., 2013; Pennington et al., 2014) have been used to bolster the performance of natural language processing systems in a wide variety of tasks, including information retrieval (Roy et al., 2018) and machine translation (Qi et al., 2018). However, approaches to learning word embeddings typically require large corpora of running text to learn high quality representations. For many languages, such resources are unavailable. This is the case for Wolastoqey, also known as Malecite-Passamaquoddy, an endangered low-resource Indigenous language. As there exist no large corpora of running text for Wolastoqey, in this paper, we leverage a bilingual dictionary to learn Wolastoqey word embeddings by encoding their corresponding English definitions into vector representations using pretrained English word and sequence representation models. Specifically, we consider representations based on pretrained word2vec (Mikolov et al., 2013), RoBERTa (Liu et al., 2019), and sentence-BERT (Reimers and Gurevych, 2019) models. We evaluate these embeddings in word prediction tasks focused on part-of-speech, animacy, and transitivity; semantic clustering; and reverse dictionary search. In all evaluations we demonstrate that approaches using these embeddings outperform task-specific baselines, without requiring any language-specific training or fine-tuning. Keywords: Word embeddings, less-resourced/endangered languages, Malecite-Passamaquoddy 1. Introduction Pretrained word embeddings (Mikolov et al., 2013; Pennington et al., 2014) have been shown to improve the performance of natural language processing (NLP) systems for a wide variety of tasks, such as machine translation (Qi et al., 2018) and information retrieval (Roy et al., 2018). However, modern approaches to learning word embeddings typically require large amounts of running text for training in order to learn high quality embeddings. Unfortunately, for many low- resource languages, such text resources do not exist in the quantity required to obtain high quality embed- dings. Malecite-Passamaquoddy (also referred to as Maliseet- Passamaquoddy and Passamaquoddy-Maliseet) is an Eastern Algonquian language spoken in regions of what is now known as New Brunswick and Que- bec, Canada, and Maine, United States. Malecite and Passamaquoddy are dialects of this language with the Malecite dialect being primarily spoken along Wolastoq (also known as the St. John River) in New Brunswick and northern Maine (Leavitt, 1996). How- ever, many speakers of the Malecite dialect use the term Wolastoqey to refer to their language. This research was carried out in Wolastokuk (i.e., on or along Wolas- toq, the territory in which the Malecite dialect is spo- ken) and the first author of this paper is wolastoqew. We therefore use the term Wolastoqey throughout this paper. There are currently approximately 300 remaining first language speakers of Wolastoqey in Canada (Statistics Canada, 2017). Due to its low-resource state, devel- oping language technologies for Wolastoqey is chal- lenging because there are no large corpora or annotated datasets available …

Content truncated to 500 words