Neural Sequence Learning Models for Word Sense Disambiguation

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1156–1167 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Neural Sequence Learning Models for Word Sense Disambiguation Alessandro Raganato, Claudio Delli Bovi and Roberto Navigli Department of Computer Science Sapienza University of Rome {raganato,dellibovi,navigli}@di.uniroma1.it Abstract Word Sense Disambiguation models ex- ist in many flavors. Even though super- vised ones tend to perform best in terms of accuracy, they often lose ground to more flexible knowledge-based solutions, which do not require training by a word expert for every disambiguation target. To bridge this gap we adopt a different perspective and rely on sequence learning to frame the disambiguation problem: we propose and study in depth a series of end-to-end neural architectures directly tailored to the task, from bidirectional Long Short-Term Memory to encoder-decoder models. Our extensive evaluation over standard bench- marks and in multiple languages shows that sequence learning enables more ver- satile all-words models that consistently lead to state-of-the-art results, even against word experts with engineered features. 1 Introduction As one of the long-standing challenges in Natural Language Processing (NLP), Word Sense Disam- biguation (Navigli, 2009, WSD) has received con- siderable attention over recent years. Indeed, by dealing with lexical ambiguity an effective WSD model brings numerous benefits to a variety of downstream tasks and applications, from Infor- mation Retrieval and Extraction (Zhong and Ng, 2012; Delli Bovi et al., 2015) to Machine Trans- lation (Carpuat and Wu, 2007; Xiong and Zhang, 2014; Neale et al., 2016). Recently, WSD has also been leveraged to build continuous vector repre- sentations for word senses (Chen et al., 2014; Ia- cobacci et al., 2015; Flekova and Gurevych, 2016). Inasmuch as WSD is described as the task of as- sociating words in context with the most suitable entries in a pre-defined sense inventory, the ma- jority of WSD approaches to date can be grouped into two main categories: supervised (or semi- supervised) and knowledge-based. Supervised models have been shown to consistently outper- form knowledge-based ones in all standard bench- marks (Raganato et al., 2017), at the expense, however, of harder training and limited flexibil- ity. First of all, obtaining reliable sense-annotated corpora is highly expensive and especially diffi- cult when non-expert annotators are involved (de Lacalle and Agirre, 2015), and as a consequence approaches based on unlabeled data and semi- supervised learning are emerging (Taghipour and Ng, 2015b; Bas¸kaya and Jurgens, 2016; Yuan et al., 2016; Pasini and Navigli, 2017). Apart from the shortage of training data, a cru- cial limitation of current supervised approaches is that a dedicated classifier (word expert) needs to be trained for every target lemma, making them less flexible and hampering their use within end- to-end applications. In contrast, knowledge-based systems do not require sense-annotated data and often draw upon the structural properties of lexico- semantic resources (Agirre et al., 2014; Moro et al., 2014; Weissenborn et al., 2015). Such sys- tems construct a model based only on the underly- ing resource, which is …

Content truncated to 500 words