Robust Handling of Polysemy via Sparse Representations

Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (*SEM), pages 265–275 New Orleans, June 5-6, 2018. c⃝2018 Association for Computational Linguistics Robust Handling of Polysemy via Sparse Representations Abhijit A. Mahabal Google amahabal@google.com Dan Roth University of Pennsylvania danroth@seas.upenn.edu Sid Mittal Google sidmittal@google.com Abstract Words are polysemous and multi-faceted, with many shades of meanings. We sug- gest that sparse distributed representations are more suitable than other, commonly used, (dense) representations to express these mul- tiple facets, and present Category Builder, a working system that, as we show, makes use of sparse representations to support multi-faceted lexical representations. We argue that the set expansion task is well suited to study these meaning distinctions since a word may be- long to multiple sets with a different reason for membership in each. We therefore exhibit the performance of Category Builder on this task, while showing that our representation captures at the same time analogy problems such as “the Ganga of Egypt” or “the Voldemort of Tolkien”. Category Builder is shown to be a more expressive lexical representation and to outperform dense representations such as Word2Vec in some analogy classes despite be- ing shown only two of the three input terms. 1 Introduction Word embeddings have received much attention lately because of their ability to represent similar words as nearby points in a vector space, thus sup- porting better generalization when comparisons of lexical items are needed, and boosting the ro- bustness achieved by some deep-learning systems. However, a given surface form often has multiple meanings, complicating this simple picture. Arora et al. (2016) showed that the vector corresponding to a polysemous term often is not close to any of that of its individual senses, thereby breaking the similar-items-map-to-nearby-points promise. The polysemy wrinkle is not merely an irritation but, in the words of Pustejovsky and Boguraev (1997), “one of the most intractable problems for language processing studies”. Our notion of Polysemy here is quite broad, since words can be similar to one another along a variety of dimensions. The following three pairs each has two similar items: (a) {ring, necklace}, (b) {ring, gang}, and (c) {ring, beep}. Note that ring is similar to all words that appear as second words in these pairs, but for different reasons, de- fined by the second token in the pairs. While this example used different senses of ring, it is easy to find examples where a single sense has multiple facets: Clint Eastwood, who is both an actor and a director, shares different aspects with directors than with actors, and Google, both a website and a major corporation, is similar to Wikipedia and General Electric along different dimensions. Similarity has typically been studied pairwise: that is, by asking how similar item A is to item B. A simple modification sharply brings to fore the issues of facets and polysemy. This modifi- cation is best viewed through the task of set ex- pansion (Wang and Cohen, 2007; Davidov et al., 2007; Jindal and Roth, 2011), which …

Content truncated to 500 words