Embeddings for Word Sense Disambiguation: An Evaluation Study

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 897–907, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Embeddings for Word Sense Disambiguation: An Evaluation Study Ignacio Iacobacci1, Mohammad Taher Pilehvar2 and Roberto Navigli1 1Department of Computer Science, Sapienza University of Rome, Italy 2Language Technology Lab, University of Cambridge, UK {iacobacci,navigli}@di.uniroma1.it mp792@cam.ac.uk Abstract Recent years have seen a dramatic growth in the popularity of word embeddings mainly owing to their ability to capture se- mantic information from massive amounts of textual content. As a result, many tasks in Natural Language Processing have tried to take advantage of the potential of these distributional models. In this work, we study how word embeddings can be used in Word Sense Disambiguation, one of the oldest tasks in Natural Language Processing and Artificial Intelligence. We propose different methods through which word embeddings can be leveraged in a state-of-the-art supervised WSD system architecture, and perform a deep analysis of how different parameters affect perfor- mance. We show how a WSD system that makes use of word embeddings alone, if designed properly, can provide significant performance improvement over a state-of- the-art WSD system that incorporates sev- eral standard WSD features. 1 Introduction Embeddings represent words, or concepts in a low-dimensional continuous space. These vec- tors capture useful syntactic and semantic infor- mation, such as regularities in language, where re- lationships are characterized by a relation-specific vector offset. The ability of embeddings to cap- ture knowledge has been exploited in several tasks, such as Machine Translation (Mikolov et al., 2013, MT), Sentiment Analysis (Socher et al., 2013), Word Sense Disambiguation (Chen et al., 2014, WSD) and Language Understanding (Mesnil et al., 2013). Supervised WSD is based on the hy- pothesis that contextual information provides a good approximation to word meaning, as sug- gested by Miller and Charles (1991): semantically similar words tend to have similar contextual dis- tributions. Recently, there have been efforts on leverag- ing embeddings for improving supervised WSD systems. Taghipour and Ng (2015) showed that the performance of conventional supervised WSD systems can be increased by taking advantage of embeddings as new features. In the same direc- tion, Rothe and Sch¨utze (2015) trained embed- dings by mixing words, lexemes and synsets, and introducing a set of features based on calculations on the resulting representations. However, none of these techniques takes full advantage of the se- mantic information contained in embeddings. As a result, they generally fail in providing substantial improvements in WSD performance. In this paper, we provide for the first time a study of different techniques for taking advantage of the combination of embeddings with standard WSD features. We also propose an effective ap- proach for leveraging embeddings in WSD, and show that this can provide significant improve- ment on multiple standard benchmarks. 2 Word Embeddings An embedding is a representation of a topologi- cal object, such as a manifold, graph, or field, in a certain space in such a way that its connectiv- ity or algebraic …

Content truncated to 500 words