Lexical Semantic Change Discovery

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6985–6998 August 1–6, 2021. ©2021 Association for Computational Linguistics 6985 Lexical Semantic Change Discovery Sinan Kurtyigit♠ Maike Park♥ Dominik Schlechtweg♠ Jonas Kuhn♠ Sabine Schulte im Walde♠ ♠Institute for Natural Language Processing, University of Stuttgart ♥Leibniz Institute for the German Language, Mannheim sinan.kurtyigit@gmail.com, park@ids-mannheim.de {schlecdk,jonas.kuhn,schulte}@ims.uni-stuttgart.de Abstract While there is a large amount of research in the field of Lexical Semantic Change Detec- tion, only few approaches go beyond a stan- dard benchmark evaluation of existing models. In this paper, we propose a shift of focus from change detection to change discovery, i.e., dis- covering novel word senses over time from the full corpus vocabulary. By heavily fine-tuning a type-based and a token-based approach on re- cently published German data, we demonstrate that both models can successfully be applied to discover new words undergoing meaning change. Furthermore, we provide an almost fully automated framework for both evaluation and discovery. 1 Introduction There has been considerable progress in Lexical Se- mantic Change Detection (LSCD) in recent years (Kutuzov et al., 2018; Tahmasebi et al., 2018; Hengchen et al., 2021), with milestones such as the first approaches using neural language mod- els (Kim et al., 2014; Kulkarni et al., 2015), the introduction of Orthogonal Procrustes alignment (Kulkarni et al., 2015; Hamilton et al., 2016), de- tecting sources of noise (Dubossarsky et al., 2017, 2019), the formulation of continuous models (Fr- ermann and Lapata, 2016; Rosenfeld and Erk, 2018; Tsakalidis and Liakata, 2020), the first uses of contextualized embeddings (Hu et al., 2019; Giu- lianelli et al., 2020), the development of solid an- notation and evaluation frameworks (Schlechtweg et al., 2018, 2019; Shoemark et al., 2019) and shared tasks (Basile et al., 2020; Schlechtweg et al., 2020). However, only a very limited amount of work applies the methods to discover novel instances of semantic change and to evaluate the usefulness of such discovered senses for external fields. That is, the majority of research focuses on the introduction of novel LSCD models, and on analyzing and evalu- ating existing models. Up to now, these preferences for development and analysis vs. application repre- sented a well-motivated choice, because the quality of state-of-the-art models had not been established yet, and because no tuning and testing data were available. But with recent advances in evaluation (Basile et al., 2020; Schlechtweg et al., 2020; Ku- tuzov and Pivovarova, 2021), the field now owns standard corpora and tuning data for different lan- guages. Furthermore, we have gained experience regarding the interaction of model parameters and modelling task (such as binary vs. graded semantic change). This enables the field to more confidently apply models to discover previously unknown se- mantic changes. Such discoveries may be useful in a range of fields (Hengchen et al., 2019; Jatowt et al., 2021), among which historical semantics and lexicography represent obvious choices (Ljubeˇsi´c, 2020). In this paper, we tune the most successful mod- …

Content truncated to 500 words