Explainable and Sparse Representations of Academic Articles for Knowledge Exploration

Proceedings of the 28th International Conference on Computational Linguistics, pages 6207–6216 Barcelona, Spain (Online), December 8-13, 2020 6207 Explainable and Sparse Representations of Academic Articles for Knowledge Exploration Keng-Te Liao†∗, Zhihong Shen§, Chiyuan Huang§, Chieh-Han Wu§, PoChun Chen†∗, Kuansan Wang§, Shou-de Lin† †National Taiwan University, §Microsoft Research d05922001@ntu.edu.tw, {zhihosh,chiyuan.huang}@microsoft.com, chiewu@microsoft.com, b02902019@ntu.edu.tw, kuansanw@microsoft.com, sdlin@csie.ntu.edu.tw Abstract We focus on a recently deployed system built for summarizing academic articles by concept tagging. The system has shown great coverage and high accuracy of concept identification which could be contributed by the knowledge acquired from millions of publications. Provided with the interpretable concepts and knowledge encoded in a pre-trained neural model, we investigate whether the tagged concepts can be applied to a broader class of applications. We propose transforming the tagged concepts into sparse vectors as representations of academic documents. The effectiveness of the representations is analyzed theoretically by a proposed framework. We also empirically show that the representations can have advantages on academic topic discovery and paper recommendation. On these applications, we reveal that the knowledge encoded in the tagging system can be effectively utilized and can help infer additional features from data with limited information. 1 Introduction Efficiently exploring knowledge is an active research topic in this era. In this work, we focus on a de- ployed system (Shen et al., 2018) which is built for summarizing academic publications via explainable concepts. An example1 of the system output is shown in Figure 1, where the system tags relevant con- cepts such as “Word2vec” and “Word order” for summarizing the given paper. Notably, the concepts are called fields-of-study (FoS). Figure 1: An example of concept (or FoS) tagging. To ensure concepts covered by various articles can be recognized, the system acquired knowledge from 170 million academic publications via deep learning models. The learned concepts are further organized by a hierarchical structure. Specifically, the hierarchy is a 6-level tree. The FoS in level 0 are the most This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http://creativecommons.org/licenses/by/4.0/. ∗ Work done during internship at Microsoft Research. 1Search result on https://academic.microsoft.com/home 6208 coarse-grained concepts such as “Computer Science” and “Chemistry”. On the contrary, FoS in level 5 are the most fine-grained concepts such as “Convolutional Deep Belief Networks” and “Phosphatase”. In the tagging process, the system would normally tag both high-level and low-level FoS for helping users efficiently and thoroughly understand ideas of the given articles. The accuracy of tagging has been carefully evaluated by human judge. However, whether the tagged FoS can be beneficial to broader applications is still unknown. We propose that this problem is worth studying as the tagging system natively possesses two advantages. The first one is the guaranteed inter- pretability. It potentially helps models leveraging the identified concepts be more explainable. Secondly, the tagging system has learned abundant knowledge from millions of academic papers. Ideally, the learned knowledge can be utilized for solving diverse natural language processing (NLP) problems as long as academic data are involved. In …

Content truncated to 500 words