Multiplex Word Embeddings for Selectional Preference Acquisition

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5247–5256, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5247 Multiplex Word Embeddings for Selectional Preference Acquisition Hongming Zhang♣∗, Jiaxin Bai♣∗, Yan Song♠, Kun Xu♥, Changlong Yu♣, Yangqiu Song♣, Wilfred Ng♣, and Dong Yu♥ ♣Department of CSE, The Hong Kong University of Science and Technology ♠Sinovation Ventures ♥Tencent AI Lab {hzhangal, jbai, cyuaq, yqsong, wilfred}@cse.ust.hk clksong@gmail.com, {kxkunxu, dyu}@tencent.com Abstract Conventional word embeddings represent words with fixed vectors, which are usu- ally trained based on co-occurrence patterns among words. In doing so, however, the power of such representations is limited, where the same word might be functionalized separately under different syntactic relations. To address this limitation, one solution is to incorporate relational dependencies of different words into their embeddings. Therefore, in this paper, we propose a multiplex word embedding model, which can be easily extended according to var- ious relations among words. As a result, each word has a center embedding to represent its overall semantics, and several relational em- beddings to represent its relational dependen- cies. Compared to existing models, our model can effectively distinguish words with respect to different relations without introducing un- necessary sparseness. Moreover, to accom- modate various relations, we use a small di- mension for relational embeddings and our model is able to keep their effectiveness. Ex- periments on selectional preference acquisi- tion and word similarity demonstrate the effec- tiveness of the proposed model, and a further study of scalability also proves that our em- beddings only need 1/20 of the original em- bedding size to achieve better performance. 1 Introduction Representing words as distributed representations is an important way for machines to process lexical semantics, which attracts much atten- tion in natural language processing (NLP) in the past few years (Mikolov et al., 2013; Penning- ton et al., 2014; Song et al., 2017, 2018b; Song and Shi, 2018) with respect to its usefulness in many downstream tasks, e.g., parsing (Chen and Manning, 2014), machine translation (Zou ∗Equal contribution. et al., 2013), coreference resolution (Lee et al., 2018), etc. Conventional word embeddings, e.g., word2vec (Mikolov et al., 2013) and GloVe (Pen- nington et al., 2014), leverage the co-occurrence information among words to train a unified em- bedding for each word. Such models are popu- lar and the resulting embeddings are widely used owing to their effectiveness and simplicity. How- ever, these embeddings are not helpful for scenar- ios requiring words functionalizing separately un- der different situations, where selectional prefer- ence (SP) (Wilks, 1975) is a typical scenario. In general, SP refers to that, given a word (pred- icate) and a dependency relation, human beings have certain preferences for the words (arguments) connecting to it. Such preferences are usually carried in dependency syntactic relations, for ex- ample, the verb ‘sing’ has plausible object words ‘song’ or ‘rhythm’ rather than other nouns such as ‘house’ or ‘potato’. With such characteristic, SP is …

Content truncated to 500 words