Dual Tensor Model for Detecting Asymmetric Lexico-Semantic Relations

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1757–1767 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Dual Tensor Model for Detecting Asymmetric Lexico-Semantic Relations Goran Glavaˇs and Simone Paolo Ponzetto Data and Web Science Group University of Mannheim B6, 26, DE-68159 Mannheim, Germany {goran, simone}@informatik.uni-mannheim.de Abstract Detection of lexico-semantic relations is one of the central tasks of computational semantics. Although some fundamental re- lations (e.g., hypernymy) are asymmetric, most existing models account for asym- metry only implicitly and use the same concept representations to support detec- tion of symmetric and asymmetric rela- tions alike. In this work, we propose the Dual Tensor model, a neural architecture with which we explicitly model the asym- metry and capture the translation between unspecialized and specialized word embed- dings via a pair of tensors. Although our Dual Tensor model needs only unspecial- ized embeddings as input, our experiments on hypernymy and meronymy detection suggest that it can outperform more com- plex and resource-intensive models. We further demonstrate that the model can ac- count for polysemy and that it exhibits sta- ble performance across languages. 1 Introduction Detection of semantic relations that hold between words is the central task of lexical semantics, tightly coupled with obtaining representations that capture meaning of words (Mikolov et al., 2013; Wieting et al., 2015; Mrkˇsi´c et al., 2016, inter alia). As such, robust detection of lexico-semantic rela- tions may benefit virtually any natural language processing application. Because lexico-semantic knowledge bases (KBs) like WordNet (Fellbaum, 1998) are general and of limited coverage, numerous methods for detect- ing lexico-semantic relations rely on distributional word representations obtained from large corpora. Although distributional models have evolved over time, from count-based (Landauer et al., 1998) and generative (Blei et al., 2003) to prediction-based (Mikolov et al., 2013), the similarity between dis- tributional vectors still indicates only the abstract semantic association and not a precise semantic re- lation (e.g., vectors of antonyms may be as similar as vectors of synonyms). Consequently, a number of approaches have been proposed for specializing distributional spaces for specific lexico-semantic relations, either by (1) modifying the learning objective or regularization of the original embedding model by incorporat- ing linguistic constraints (Yu and Dredze, 2014; Kiela et al., 2015) or (2) retroactively fitting the pre-trained unspecialized embeddings to linguis- tic constraints (Faruqui et al., 2015; Mrkˇsi´c et al., 2016). However, these methods specialize distribu- tional vector spaces primarily for detecting the sym- metric relation of semantic similarity (i.e., graded synonymy) and not for asymmetric lexico-semantic relations such as hypernymy and meronymy. On the other hand, models for embedding KBs (Bordes et al., 2013; Socher et al., 2013; Yang et al., 2015) uniformly model both symmetric and asymmetric relations. They learn a single vector representation (i.e., embedding) for each KB concept, assuming implicitly that the same concept representation is equally useful for predicting symmetric and asym- metric relations alike. Relation-specific learning-based models have, to the largest extent, targeted hypernymy. Distri- butional models predict the …

Content truncated to 500 words