An Evaluation Method for Diachronic Word Sense Induction

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3171–3180 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3171 An Evaluation Method for Diachronic Word Sense Induction Ashjan Alsulaimani School of Computer Science and Statistics & Trinity Centre for Computing and Language Studies Trinity College Dublin alsulaia@tcd.ie Erwan Moreau School of Computer Science and Statistics & Adapt Centre Trinity College Dublin moreaue@scss.tcd.ie Carl Vogel School of Computer Science and Statistics & Trinity Centre for Computing and Language Studies Trinity College Dublin vogel@scss.tcd.ie Abstract The task of Diachronic Word Sense Induc- tion (DWSI) aims to identify the meaning of words from their context, taking the tempo- ral dimension into account. In this paper we propose an evaluation method based on large- scale time-stamped annotated biomedical data, and a range of evaluation measures suited to the task. The approach is applied to two re- cent DWSI systems, thus demonstrating its rel- evance and providing an in-depth analysis of the models. 1 Introduction Words naturally evolve through time, their mean- ing may encounter subtle or radical changes result- ing in a variety of senses. For example, the word mouse only had the meaning of animal until it acquired a brand new sense in 1980 as computer device. But sense changes are not always so defi- nite, a word usage may drift progressively from its original sense or be affected by historical events. A recent example of this phenomenon is the word coronavirus, which has seen a dramatic us- age surge in 2020 because of the emergence of its SARS-CoV-2 variant. Before 2020, the word coronavirus was mostly a technical term de- scribing a family of viruses, but it is now used in the mainstream media to mean the specific SARS-CoV- 2, the related Covid19 disease or even the general health crisis and its consequences. The dynamic behaviour of words contributes to semantic ambiguity, which is a challenge in many NLP tasks. The ability to detect such changes across time could potentially benefit various ap- plications, such as machine translation and infor- mation retrieval. In the biomedical domain, it can improve the quality of the automatic identification of senses in contexts where no complete terminol- ogy is available, such as with clinical notes, and to assist indexers who build terminology resources. Recent research focused on detecting seman- tic shifts across time (Kutuzov et al., 2018) but also Diachronic Word Sense Induction (Emms and Kumar Jayapal, 2016). The task of Diachronic Word Sense Induction (DWSI) is similar to Word Sense Induction (WSI) in identifying the meaning of words from their context, but also takes the tem- poral dimension into account. In §2 we briefly present two Bayesian models that have been proposed for the DWSI task: Emms and Kumar Jayapal (2016) proposed a model which represents the evolution of word senses in order to detect the emergence year of new senses. A different model was proposed by Frermann and Lapata (2016), focusing instead on capturing the subtle meaning changes within a sense …

Content truncated to 500 words