SensEmbed: Learning Sense Embeddings for Word and Relational Similarity

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 95–105, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics SENSEMBED: Learning Sense Embeddings for Word and Relational Similarity Ignacio Iacobacci, Mohammad Taher Pilehvar and Roberto Navigli Department of Computer Science Sapienza University of Rome {iacobacci,pilehvar,navigli}@di.uniroma1.it Abstract Word embeddings have recently gained considerable popularity for modeling words in different Natural Language Processing (NLP) tasks including seman- tic similarity measurement. However, notwithstanding their success, word embeddings are by their very nature unable to capture polysemy, as different meanings of a word are conflated into a single representation. In addition, their learning process usually relies on massive corpora only, preventing them from taking advantage of structured knowledge. We address both issues by proposing a multi- faceted approach that transforms word embeddings to the sense level and lever- ages knowledge from a large semantic network for effective semantic similarity measurement. We evaluate our approach on word similarity and relational similar- ity frameworks, reporting state-of-the-art performance on multiple datasets. 1 Introduction The much celebrated word embeddings represent a new branch of corpus-based distributional se- mantic model which leverages neural networks to model the context in which a word is expected to appear. Thanks to their high coverage and their ability to capture both syntactic and semantic in- formation, word embeddings have been success- fully applied to a variety of NLP tasks, such as Word Sense Disambiguation (Chen et al., 2014), Machine Translation (Mikolov et al., 2013b), Re- lational Similarity mantic Relatedness(Mikolov et al., 2013c), Se- (Baroni et al., 2014) and Knowledge Representation (Bordes et al., 2013). However, word embeddings inherit two im- portant limitations from their antecedent corpus- based distributional models: (1) they are unable to model distinct meanings of a word as they conflate the contextual evidence of different meanings of a word into a single vector; and (2) they base their representations solely on the distributional statis- tics obtained from corpora, ignoring the wealth of information provided by existing semantic re- sources. Several research works have tried to address these problems. For instance, basing their work on the original sense discrimination approach of Reisinger and Mooney (2010), Huang et al. (2012) applied K-means clustering to decompose word embeddings into multiple prototypes, each denot- ing a distinct meaning of the target word. How- ever, the sense representations obtained are not linked to any sense inventory, a mapping that con- sequently has to be carried out either manually, or with the help of sense-annotated data. Another line of research investigates the possibility of tak- ing advantage of existing semantic resources in word embeddings. A good example is the Relation Constrained Model (Yu and Dredze, 2014). When computing word embeddings, this model replaces the original co-occurrence clues from text corpora with the relationship information derived from the Paraphrase Database1 (Ganitkevitch et al., 2013, PPDB), an automatically extracted dataset of para- phrase pairs. However, none of these techniques have simul- taneously solved …

Content truncated to 500 words