Cluster Labeling by Word Embeddings and WordNet's Hypernymy

Cluster Labeling by Word Embeddings and WordNet’s Hypernymy Hanieh Poostchi University of Technology Sydney Capital Markets CRC hpoostchi@cmcrc.com Massimo Piccardi University of Technology Sydney massimo.piccardi@uts.edu.au Abstract Cluster labeling is the assignment of rep- resentative labels to clusters of documents or words. Once assigned, the labels can play an important role in applications such as navigation, search and document clas- sification. However, finding appropriately descriptive labels is still a challenging task. In this paper, we propose various approaches for assigning labels to word clusters by leveraging word embeddings and the synonymy and hypernymy rela- tions in the WordNet lexical ontology. Ex- periments carried out using the WebAP document dataset have shown that one of the approaches stand out in the compari- son and is capable of selecting labels that are reasonably aligned with those chosen by a pool of four human annotators. 1 Introduction and Related Work Document collections are often organized into clusters of either documents or words to facilitate applications such as navigation, search and classi- fication. The organization can prove more useful if its clusters are characterized by sets of represen- tative labels. The task of assigning a set of labels to each individual cluster in a document organi- zation is known as cluster labeling (Wang et al., 2014) and it can provide a useful description of the collection in addition to fundamental support for navigation and search. In Manning et al. (2008), cluster labeling ap- proaches have been subdivided into i) differen- tial cluster labeling and ii) cluster-internal label- ing. The former selects cluster labels by compar- ing the distribution of terms in one cluster with those of the other clusters while the latter selects labels that are solely based on each cluster indi- vidually. Cluster-internal labeling approaches in- clude computing the clusters’ centroids and using them as labels, or using lists of terms with high- est frequencies in the clusters. However, all these approaches can only select cluster labels from the terms and phrases that explicitly appear in the doc- uments, possibly failing to provide an appropri- ate level of abstraction or description (Lau et al., 2011). As an example, a word cluster containing words dog and wolf should not be labeled with ei- ther word, but as canids. For this reason, in this paper we explore several approaches for labeling word clusters obtained from a document collec- tion by leveraging the synonymy and hypernymy relations in the WordNet taxonomy (Miller, 1995), together with word embeddings (Mikolov et al., 2013; Pennington et al., 2014). A hypernymy relation represents an asymmetric relation between a class and each of its instances. A hypernym (e.g., vertebrate) has a broader con- text than its hyponyms (bird, fishes, reptiles etc). Conversely, the contextual properties of the hy- ponyms are usually a subset of those of their hy- pernym(s). Hypernymy has been used extensively in natural language processing, including in re- cent works such as Yu et al. (2015) and HyperVec (Nguyen et al., 2017) that have proposed learning word embeddings that …

Content truncated to 500 words