Exploring Bilingual Word Embeddings for Hiligaynon, a Low-Resource Language

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2573–2580 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2573 Exploring Bilingual Word Embeddings for Hiligaynon, a Low-Resource Language Leah Michel, Viktor Hangya, Alexander Fraser Center for Information and Language Processing LMU Munich, Germany l.michel@campus.lmu.de, {hangyav, fraser}@cis.lmu.de Abstract This paper investigates the use of bilingual word embeddings for mining Hiligaynon translations of English words. There is very little research on Hiligaynon, an extremely low-resource language of Malayo-Polynesian origin with over 9 million speakers in the Philippines (we found just one paper). We use a publicly available Hiligaynon corpus with only 300K words, and match it with a comparable corpus in English. As there are no bilingual resources available, we manually develop a English-Hiligaynon lexicon and use this to train bilingual word embeddings. But we fail to mine accurate translations due to the small amount of data. To find out if the same holds true for a related language pair, we simulate the same low-resource setup on English to German and arrive at similar results. We then vary the size of the comparable English and German corpora to determine the minimum corpus size necessary to achieve competitive results. Further, we investigate the role of the seed lexicon. We show that with the same corpus size but with a smaller seed lexicon, performance can surpass results of previous studies. We release the lexicon of 1,200 English-Hiligaynon word pairs we created to encourage further investigation. Keywords: bilingual word embeddings, bilingual lexicon induction, post-hoc mapping, low-resource languages, Hiligaynon 1. Introduction Since the introduction of Skip-gram and Continuous Bag- of-Words (Mikolov et al., 2013a) followed by the release of GloVe (Pennington et al., 2014), the use of word em- beddings, the continuous vector representations of words, has become the norm for many natural language process- ing (NLP) tasks. From sentence classification (Kim, 2014), part-of-speech tagging (Abka, 2016), named entity recog- nition (Melamud et al., 2016), to sentiment analysis (Ruder et al., 2016), it is hard to imagine modern NLP without word embeddings. How this success of monolingual word embeddings (MWEs) can extend to a bilingual setup – to represent meaning and transfer knowledge in bilingual tasks – sparks interests that paved the way for investigating different models of bilingual word embeddings (BWEs). When two MWEs share a single vector space in the form of BWEs, it is likely that, following the concept of MWEs correlating distance with semantic similarity, the target lan- guage word closest to the source language word is the trans- lation equivalent. To evaluate the quality of BWEs, this paper conducts exper- iments with the task of bilingual lexicon induction (BLI). This task aims to mine accurate word translations from a source language to a target language. The output is a list of source words with corresponding lists of top-n transla- tions in the target language based on cosine similarity in the BWE space. Unlike translations that are learned from parallel corpora which are …

Content truncated to 500 words