Modeling Semantic Compositionality with Sememe Knowledge

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5706–5715 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5706 Modeling Semantic Compositionality with Sememe Knowledge Fanchao Qi1∗, Junjie Huang2∗†, Chenghao Yang3†, Zhiyuan Liu1, Xiao Chen4, Qun Liu4, Maosong Sun1‡ 1Department of Computer Science and Technology, Tsinghua University Institute for Artificial Intelligence, Tsinghua University State Key Lab on Intelligent Technology and Systems, Tsinghua University 2School of ASEE, Beihang University 3Software College, Beihang Unviersity 4Huawei Noah’s Ark Lab qfc17@mails.tsinghua.edu.cn, {hjj1997,alanyang}@buaa.edu.cn {liuzy,sms}@tsinghua.edu.cn, {chen.xiao2,qun.liu}@huawei.com Abstract Semantic compositionality (SC) refers to the phenomenon that the meaning of a complex linguistic unit can be composed of the mean- ings of its constituents. Most related works focus on using complicated compositional- ity functions to model SC while few works consider external knowledge in models. In this paper, we verify the effectiveness of se- memes, the minimum semantic units of hu- man languages, in modeling SC by a confir- matory experiment. Furthermore, we make the first attempt to incorporate sememe knowl- edge into SC models, and employ the sememe- incorporated models in learning representa- tions of multiword expressions, a typical task of SC. In experiments, we implement our mod- els by incorporating knowledge from a famous sememe knowledge base HowNet and perform both intrinsic and extrinsic evaluations. Exper- imental results show that our models achieve significant performance boost as compared to the baseline methods without considering se- meme knowledge. We further conduct quan- titative analysis and case studies to demon- strate the effectiveness of applying sememe knowledge in modeling SC. All the code and data of this paper can be obtained on https: //github.com/thunlp/Sememe-SC. 1 Introduction Semantic compositionality (SC) is defined as the linguistic phenomenon that the meaning of a syn- tactically complex unit is a function of meanings of the complex unit’s constituents and their combi- nation rule (Pelletier, 1994). Some linguists regard SC as the fundamental truth of semantics (Pel- letier, 2016). In the field of NLP, SC has proved effective in many tasks including language model- ∗Indicates equal contribution †Work done during internship at Tsinghua University ‡Corresponding author ing (Mitchell and Lapata, 2009), sentiment analy- sis (Maas et al., 2011; Socher et al., 2013b), syn- tactic parsing (Socher et al., 2013a), etc. Most literature on SC pays attention to using vector-based distributional models of semantics to learn representations of multiword expressions (MWEs), i.e., embeddings of phrases or com- pounds. Mitchell and Lapata (2008) conduct a pi- oneering work in which they introduce a general framework to formulate this task: p = f(w1, w2, R, K)1, (1) where f is the compositionality function, p de- notes the embedding of an MWE, w1 and w2 rep- resent the embeddings of the MWE’s two con- stituents, R stands for the combination rule and K refers to the additional knowledge which is needed to construct the semantics of the MWE. Among the proposed approaches for this task, most of them ignore R and K, centering on re- forming compositionality function …

Content truncated to 500 words