Measure and Evaluation of Semantic Divergence across Two Languages

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1247–1258 August 1–6, 2021. ©2021 Association for Computational Linguistics 1247 Measure and Evaluation of Semantic Divergence across Two Languages Syrielle Montariol ∗ INRIA Paris syrielle.montariol@inria.fr Alexandre Allauzen ESPCI Paris Dauphine University alexandre.allauzen@espci.psl.eu Abstract Languages are dynamic systems: word usage may change over time, reflecting various so- cietal factors. However, all languages do not evolve identically: the impact of an event, the influence of a trend or thinking, can differ be- tween communities. In this paper, we pro- pose to track these divergences by compar- ing the evolution of a word and its translation across two languages. We investigate several methods of building time-varying and bilin- gual word embeddings, using contextualised and non-contextualised embeddings. We pro- pose a set of scenarios to characterize semantic divergence across two languages, along with a setup to differentiate them in a bilingual cor- pus. We evaluate the different methods by gen- erating a corpus of synthetic semantic change across two languages, English and French, be- fore applying them to newspaper corpora to detect bilingual semantic divergence and pro- vide qualitative insight for the task. We con- clude that BERT embeddings coupled with a clustering step lead to the best performance on synthetic corpora; however, the performance of CBOW embeddings is very competitive and more adapted to an exploratory analysis on a large corpus. 1 Introduction Languages evolve throughout time: for many words, their usages along with their frequent collo- cations and associations can change, revealing the evolution of the society (Aitchison, 2001). How- ever, all languages do not evolve identically: the im- pact of an event, the influence of a trend or thinking, can differ between communities. Moreover, lan- guages do not evolve independently; some words can be inherited and borrowed between languages. For example, cognates — words that have the same ∗ This work was carried out while the author was working at LISN-CNRS. etymological origin and similar meaning in two lan- guages — can sometimes diverge into false friends, due to particular features of one language and its associated culture and history. A more specific example is the Russian word “ukrop”, meaning “dill”. It started to be used by Russian people as an ethnic slur—a pejorative term—to talk about Ukrainian soldiers at the be- ginning of the Russian-Ukrainian conflict (Stewart et al., 2017). Then, Ukrainian people started to use it to designate their own patriots, in a positive way. Analysing the evolution of this word can lead to a better understanding of the evolution of the conflict; on the contrary, without suitable tools and methods to detect the divergence in its usage and connotation between communities, one might draw spurious results when analysing texts of this period. Diachronic semantic change detection is an emerging field in Natural Language Processing, building upon the growing number of digitised texts with temporal metadata publicly available in various languages. It opens new perspectives of …

Content truncated to 500 words