De-Conflated Semantic Representations

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1680–1690, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics De-Conflated Semantic Representations Mohammad Taher Pilehvar and Nigel Collier Language Technology Lab Department of Theoretical and Applied Linguistics University of Cambridge Cambridge, UK {mp792,nhc30}@cam.ac.uk Abstract One major deficiency of most semantic repre- sentation techniques is that they usually model a word type as a single point in the semantic space, hence conflating all the meanings that the word can have. Addressing this issue by learning distinct representations for individ- ual meanings of words has been the subject of several research studies in the past few years. However, the generated sense representations are either not linked to any sense inventory or are unreliable for infrequent word senses. We propose a technique that tackles these prob- lems by de-conflating the representations of words based on the deep knowledge that can be derived from a semantic network. Our ap- proach provides multiple advantages in com- parison to the previous approaches, including its high coverage and the ability to generate accurate representations even for infrequent word senses. We carry out evaluations on six datasets across two semantic similarity tasks and report state-of-the-art results on most of them. 1 Introduction Modeling the meanings of linguistic items in a machine-interpretable form, i.e., semantic represen- tation, is one of the oldest, yet most active, areas of research in Natural Language Processing (NLP). The field has recently experienced a resurgence of interest with neural network-based models that view the representation task as a language modeling prob- lem and learn dense representations (usually re- ferred to as embeddings) by efficiently processing massive amounts of texts. However, either in its conventional count-based form (Turney and Pantel, 2010) or the recent predictive approach, the prevail- ing objective of representing each word type as a single point in the semantic space has a major limita- tion: it ignores the fact that words can have multiple meanings and conflates all these meanings into a sin- gle representation. This objective can have negative impacts on accurate semantic modeling, e.g., seman- tically unrelated words that are synonymous to dif- ferent senses of a word are pulled towards each other in the semantic space (Neelakantan et al., 2014). For example, the two semantically-unrelated words squirrel and keyboard are pulled towards each other in the semantic space for their similarities to two dif- ferent senses of mouse, i.e., rodent and computer in- put device. Recently, there has been a growing interest in ad- dressing the meaning conflation deficiency of word representations. A series of techniques have been developed to associate a word to multiple points in the semantic space by clustering its contexts in a given text corpus and learning distinct rep- resentations for individual clusters (Reisinger and Mooney, 2010; Huang et al., 2012). However, these techniques usually assume a fixed number of word senses per word type, disregarding the fact that the number of senses per word can range from one (monosemy) …

Content truncated to 500 words