Semi-Supervised and Unsupervised Sense Annotation via Translations

Proceedings of Recent Advances in Natural Language Processing, pages 504–513 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_057 504 Semi-Supervised and Unsupervised Sense Annotation via Translations Bradley Hauer, Grzegorz Kondrak, Yixing Luan, Arnob Mallik, Lili Mou Alberta Machine Intelligence Institute, Department of Computing Science University of Alberta, Edmonton, Canada {bmhauer,gkondrak,yixing1,amallik,lmou}@ualberta.ca Abstract Acquisition of multilingual training data con- tinues to be a challenge in word sense dis- ambiguation (WSD). To address this prob- lem, unsupervised approaches have been pro- posed to automatically generate sense anno- tations for training supervised WSD systems. We present three new methods for creating sense-annotated corpora which leverage trans- lations, parallel bitexts, lexical resources, as well as contextual and synset embeddings. Our semi-supervised method applies machine translation to transfer existing sense annota- tions to other languages. Our two unsuper- vised methods refine sense annotations pro- duced by a knowledge-based WSD system via lexical translations in a parallel corpus. We ob- tain state-of-the-art results on standard WSD benchmarks. 1 Introduction Word sense disambiguation, the task of identifying the meaning of a word in context, is one of the central problems in natural language understanding (Navigli, 2018). It is a well-studied benchmark for evaluating contextualized representations of words (Loureiro et al., 2021), and is better understood than tasks such as WiC (Pilehvar and Camacho- Collados, 2019). Modern WSD methods can be divided into supervised and knowledge-based ap- proaches. The former depend on sense-annotated corpora, such as SemCor (Miller et al., 1994), while the latter rely instead on semantic knowledge bases such as WordNet (Miller, 1995). While supervised WSD systems typically out- perform knowledge-based systems (Scarlini et al., 2020b), their utility is limited by the availability of sufficiently large sense-annotated corpora for training. This includes systems based on contextu- alized embeddings (Bevilacqua and Navigli, 2020). In particular, there is a severe lack of high-quality sense-annotated corpora for languages other than English. This limitation has motivated the devel- opment of methods aimed at automatically disam- biguating a large number of word tokens in a given unannotated corpus, ideally covering a wide range of word and sense types, while minimizing noise (Pasini and Navigli, 2017; Scarlini et al., 2019; Barba et al., 2020). The automatically tagged cor- pus can then be used to train a supervised WSD system, satisfying the dependency on training data without the need for manual annotation. Following recent theoretical work Hauer and Kondrak (2020) on establishing the semantic equiv- alence of mutual translations, we introduce three translation-based methods for generating sense- tagged corpora. All three methods make use of lexical knowledge bases, and semantic informa- tion obtained from word-level translations. Semi- supervised LABELPROP creates a synthetic par- allel corpus (bitext) by applying machine trans- lation to a monolingual manually-annotated cor- pus, and projecting annotations to the target lan- guage. Similarly, unsupervised LABELGEN ap- plies a knowledge-based WSD system to the En- glish side of a bitext, and projects the resulting sense annotations across bitexts onto other lan- guages. Finally, unsupervised LABELSYNC pro- duces sense-annotated corpora in two languages at once by independently …

Content truncated to 500 words