GenSense: A Generalized Sense Retrofitting Model

Proceedings of the 27th International Conference on Computational Linguistics, pages 1662–1671 Santa Fe, New Mexico, USA, August 20-26, 2018. 1662 GenSense: A Generalized Sense Retrofitting Model Yang-Yin Lee1*, Ting-Yu Yen1*, Hen-Hsen Huang1, Yow-Ting Shiue1, and Hsin-Hsi Chen1,2 1Department of Computer Science and Information Engineering National Taiwan University No. 1, Sec. 4, Roosevelt Road, Taipei, 10617 Taiwan 2MOST Joint Research Center for AI Technology and All Vista Healthcare, Taiwan {yylee, tyyen, hhhuang}@nlg.csie.ntu.edu.tw, orina1123@gmail.com, hhchen@ntu.edu.tw Abstract With the aid of recently proposed word embedding algorithms, the study of semantic similarity has progressed and advanced rapidly. However, many natural language processing tasks need sense level representation. To address this issue, some researches propose sense embedding learning algorithms. In this paper, we present a generalized model from the existing sense retrofitting model. The generalization takes three major components: semantic relations between the senses, the relation strength and the semantic strength. In the experiments, we show that the generalized model outperforms the previous approaches in three aspects: semantic relatedness, contextual word similarity and semantic difference. 1 Introduction The distributed representation of word model (word embedding) has drawn great interest in recent years due to its ability to acquire syntactic and semantic information from a large unannotated corpus (Mikolov et al., 2013; Pennington et al., 2014). With the pre-trained word embedding, some researches propose post-processing models that incorporate with the existing semantic knowledge into the word embedding model (Faruqui et al., 2015; Yu and Dredze, 2014). However, word embedding models use only one vector to represent a word, and are problematic in some natural language processing applications that require sense level representation (e.g., word sense disambiguation, semantic relation identification, etc.). As a result, some researches try to resolve the polysemy and homonymy issue and introduce sense level embedding, either act as pre-process (Iacobacci et al., 2015) or post-process (Jauhar et al., 2015) fashion. In this research, we focus on the post-processing sense retrofitting approach and propose GenSense, a generalized sense embedding learning framework that retrofits a pre-trained word embedding via incorporating with the semantic relations between the senses, the relation strength and the semantic strength. Although some parts of the idea are not new, it is the first time of putting all the parts into a generalized framework. Our proposed GenSense for generating low-dimensional sense embedding is inspired from sense retro (Jauhar et al., 2015), but has three major differences. First, we generalize the semantic relations from positive relations (e.g., synonyms, hyponyms, paraphrase, etc.) to positive and negative relations (e.g., antonyms). Second, each relation incorporates with both the semantic strength and the relation strength. Within a semantic relation, there should be a weighting for each semantic strength. For example, although jewel has the synonyms gem and rock, it is clear that the similarity between (jewel, gem) is higher than (jewel, rock), and thus (jewel, gem) should have higher weight. Last, GenSense gives different relations with different relation strengths. For example, if the objective is to train a sense embedding that can distinguish between the positive and …

Content truncated to 500 words