Homonym normalisation by word sense clustering: a case in Japanese

Proceedings of the 28th International Conference on Computational Linguistics, pages 3324–3332 Barcelona, Spain (Online), December 8-13, 2020 3324 Homonym normalisation by word sense clustering: a case in Japanese Yo Sato Satoama Language Services Kingson Upon Thames, U.K. yo@satoama.co.uk Kevin Heffernan Kwansei Gakuin University Sanda, Hyogo, Japan kevin@kwansei.ac.jp Abstract This work presents a method of word sense clustering that differentiates homonyms and merge homophones, taking Japanese as an example, where orthographical variation causes problem for language processing. It uses contextualised embeddings (BERT) to cluster tokens into distinct sense groups, and we use these groups to normalise synony- mous instances to a single representative form. We see the benefit of this normalisation in language model, as well as in transliteration. 1 Introduction This work presents a method of clustering homonyms that uses contextualised word embed- dings, assesses its performance for polysemy detection and shows two use cases in Japanese, normalisation and transliteration. Japanese has multi-typed orthography, using three different alphabets, which creates challenges as homonyms/homophones are extensively observed. Our main proposal is to detect semantically equivalent groups in them, thereby cerating normalised, semantically less noisy data in pre-processing, to the benefit of language modelling. We also apply the same technique to a long-standing ‘annoyance’ in Japanese input, i.e. transliteration process. Homonymy is a universal phenomenon, and distinguishing their distinct meanings is a major problem in language processing. The advent of contextualised word embeddings (Peters et al., 2018; Devlin et al., 2018) has made it possible to differentiate token level occurrences, and they have indeed been utilised for supervised word sense disambiguation (Huang et al., 2019), but whether it can also capture latent meaning distinctions without labels is unclear, especially when the meaning boundaries are vague (Mickus et al., 2020). Our strategy is to detect relatively clear-cut cases by using a halting criterion on the number of clusters. Japanese offers a unique platform to conduct such clustering on, since along with homonyms, their opposite counterpart, heterographs —words semantically and phonemically identical but written differently— systematically coexist. We will show first that accurate enough clustering is possible via contextualised embeddings and then that the clustering improves the two aspects mentioned above, normalisation and transliteration. 2 Terminology Terminology in which to describe homonymy and heterography and related phenomena is rather confusing and often confused, and some tidying-up is in order for clarity. We have at least six related terms, homo-/hetero- prefixes with -phone/-nym/-graph suffixes that describe a rela- tion that holds in a set (mostly pair) of word types. In accordance with how the terms are conventionally used, we could understand these roughly to correspond to the combinations of the identity/difference in three parameters, sound, writing and meaning: the combination of same sound, same spelling but different meaning is for homonyms and so on. Leaving out the This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http: //creativecommons.org/licenses/by/4.0/. 3325 uninteresting combinations of the same and different in all these respects, we are left with six combinations. The following table shows …

Content truncated to 500 words