Exploring Semantic Capacity of Terms

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8509–8518, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8509 Exploring Semantic Capacity of Terms Jie Huang∗,1,4 Zilong Wang∗,2 Kevin Chen-Chuan Chang1,4 Wen-mei Hwu1,4 Jinjun Xiong3,4 1University of Illinois at Urbana-Champaign, USA 2University of California at San Diego, USA 3IBM Thomas J. Watson Research Center, USA 4IBM-Illinois Center for Cognitive Computing Systems Research (C3SR), USA {jeffhj, kcchang, w-hwu}@illinois.edu zlwang@ucsd.edu, jinjun@us.ibm.com Abstract We introduce and study semantic capacity of terms. For example, the semantic capacity of artificial intelligence is higher than that of linear regression since artificial intelligence possesses a broader meaning scope. Under- standing semantic capacity of terms will help many downstream tasks in natural language processing. For this purpose, we propose a two-step model to investigate semantic capac- ity of terms, which takes a large text corpus as input and can evaluate semantic capacity of terms if the text corpus can provide enough co- occurrence information of terms. Extensive ex- periments in three fields demonstrate the effec- tiveness and rationality of our model compared with well-designed baselines and human-level evaluations. 1 Introduction Terms are not all considered equal. For instance, in computer science, the meaning scope of artificial intelligence or computer architecture is broader than that of linear regression. To study this phe- nomenon, in this paper, we introduce Semantic Capacity, which is a scalar value to characterize the meaning scope of a term. A good command of semantic capacity will give us more insight into the granularity of terms and allow us to describe things more precisely, which is a crucial step for down- stream tasks such as keyword extraction (Hulth, 2003; Beliga et al., 2015; Firoozeh et al., 2020) and semantic analysis (Landauer et al., 1998; Goddard, 2011). Figure 1 shows the fingerprint visualization of a computer scientist, which is generated by Elsevier Fingerprint Engine1, a popular system that creates *Asterisk indicates equal contribution. Work done while visiting University of Illinois at Urbana-Champaign. 1https://www.elsevier.com/solutions/ elsevier-fingerprint-engine an index of weighted terms for research profiling. From the example, we can find that there exist some non-ideal terms, such as learning whose seman- tic capacity is too high, backpropagation whose semantic capacity is too low, and even irrelevant terms such as color. Understanding semantic ca- pacity of terms will help us to choose better terms to describe entities. Besides, combining with other techniques like word similarity, semantic capacity can also help keyword replacement. For instance, to describe the computer scientist depicted in Fig- ure 1, if the audience is a layman of computer science, we should use terms with high semantic capacity like artificial intelligence. But for an ex- pert in the corresponding domain, we can select terms with low semantic capacity like object recog- nition to make the fingerprint more precise. However, there are countless terms in human language, which means that it is extremely hard to investigate semantic capacity for all existing terms. Besides, semantic capacity of terms is also ambigu- ous in different domains. …

Content truncated to 500 words