RuSemShift: a dataset of historical lexical semantic change in Russian

Proceedings of the 28th International Conference on Computational Linguistics, pages 1037–1047 Barcelona, Spain (Online), December 8-13, 2020 1037 RuSemShift: a dataset of historical lexical semantic change in Russian Julia Rodina National Research University Higher School of Economics Moscow, Russia julia.rodina97@gmail.com Andrey Kutuzov University of Oslo Oslo, Norway andreku@ifi.uio.no Abstract We present RuSemShift, a large-scale manually annotated test set for the task of semantic change modeling in Russian for two long-term time period pairs: from the pre-Soviet through the So- viet times and from the Soviet through the post-Soviet times. Target words were annotated by multiple crowd-source workers. The annotation process was organized following the DURel framework and was based on sentence contexts extracted from the Russian National Corpus. Additionally, we report the performance of several distributional approaches on RuSemShift, achieving promising results, which at the same time leave room for other researchers to improve. 1 Introduction Language is a constantly changing system by its nature, since it is a method of communication and a social instrument. As such, it should meet the needs of the speakers, and thus it adapts to changes in the society and the ever-changing world. As a part of language, lexical meaning also evolves over time, with words undergoing diachronic (or temporal) semantic shifts (Traugott and Dasher, 2001). Tracing semantic change can be important either in itself, as a linguistic study, or for practical down- stream applications, for example, in socio-linguistic research. Manual analysis of such shifts is time- consuming and laborious even after the emergence of large representative corpora, since one needs to look through a lot of examples and lexicographic resources which often do not record current lexical changes in language due to limited resources. Thus, researchers are trying to model these processes using advanced computational approaches often based on distributional semantics and dense word em- beddings (Kutuzov et al., 2018; Tang, 2018). However, this is still mostly done for English: often simply because manually annotated test data is not available for other languages. Recently, consistently annotated lexical semantic change test sets for multiple languages started to appear; see, for example, Schlechtweg et al. (2020). In this paper, we continue this vein of work by presenting RuSemShift. RuSemShift1 is the first historical semantic change dataset for Russian annotated according to the DURel framework (Schlechtweg et al., 2018) using a large crowd-sourcing platform, instead of personal intuitions of individual researchers. It allows to evaluate semantic change detection systems by their ability to estimate the shifts which occurred to Russian words either after 1917 (the fall of the Russian Empire) or after 1990 (the fall of the Soviet Union). The rest of the paper is organized as follows: in Section 2, we put our research in the context of the related work. In Section 3, we present the employed corpora and the process of the dataset creation. Section 4 describes the annotation itself. In Section 5, we empirically evaluate several existing semantic change detection algorithms (based on static and contextualized embeddings) on RuSemShift to check its …

Content truncated to 500 words