A Generalized Method for Automated Multilingual Loanword Detection

Proceedings of the 29th International Conference on Computational Linguistics, pages 4996–5013 October 12–17, 2022. 4996 A Generalized Method for Automated Multilingual Loanword Detection Abhijnan Nath∗, Sina Mahdipour Saravani∗†, Ibrahim Khebour, Sheikh Mannan, Zihui Li and Nikhil Krishnaswamy Situated Grounding and Natural Language (SIGNAL) Lab Department of Computer Science, Colorado State University Fort Collins, CO, USA abhijnan.nath@colostate.edu,sina@cs.utah.edu†, {ibrahim.khebour,sheikh.mannan,nkrishna}@colostate.edu Abstract Loanwords are words incorporated from one language into another without translation. Sup- pose two words from distantly-related or unre- lated languages sound similar and have a sim- ilar meaning. In that case, this is evidence of likely borrowing. This paper presents a method to automatically detect loanwords across vari- ous language pairs, accounting for differences in script, pronunciation and phonetic transfor- mation by the borrowing language. We incorpo- rate edit distance, semantic similarity measures, and phonetic alignment. We evaluate on 12 lan- guage pairs and achieve performance compara- ble to or exceeding state of the art methods on single-pair loanword detection tasks. We also demonstrate that multilingual models perform the same or often better than models trained on single language pairs and can potentially gen- eralize to unseen language pairs with sufficient data, and that our method can exceed human performance on loanword detection. 1 Introduction Throughout history, words and phrases have been exchanged between languages around the world (Weinreich, 1954). This can obscure genetic re- lations between languages (e.g., many people er- roneously believe English and French are more closely related than they are) but may also increase comprehension of foreign languages by monoglots (e.g., written French is often partially comprehensi- ble by English speakers). As Zhang et al. (2021) observe, detecting that a word is loanword is conceptually straightforward: both similar sound and meaning suggests too great a coincidence for different words to have converged by chance1. Detecting loanwords computationally *These authors contributed equally to this work. †This work performed at Colorado State University. 1There are exceptions, e.g., Persian bad vs. English “bad” and Mbabaram dog/dúg vs. English “dog”. These are beyond the scope of this paper. has therefore relied on pairwise similarity mea- sures based on transliteration detection and edit distance. However, foundational work in linguistic borrowing, e.g., by Haugen (1950) and Betz (1959), established that when borrowing words into a recip- ient language, speakers of that language will repro- duce existing linguistic patterns when using new words, and the patterns that recipient speakers im- pose upon a borrowed word vary across time (Köll- ner and Dellert, 2016), and language pairs. Some languages may adopt a word without much pho- netic change due to already-similar phonotactics. Others may fit imported words into a rigid sound pattern, with sometimes significant transformation. Still others may change the meaning. Changes are particular to the language pair, so automatically de- tecting loanwords between arbitrary languages is challenging. However, if successful, such capabili- ties would also provide benefits to many other NLP tasks such as machine translation, coreference, and named-entity recognition (NER), because common vocabulary, coreferents, or named entities across languages may often be …

Content truncated to 500 words