Dual Attention Model for Citation Recommendation

Dual Attention Model for Citation Recommendation Yang Zhang Graduate School of Informatics Kyoto University zhang.yang.33z@st.kyoto-u.ac.jp Qiang Ma Graduate School of Informatics Kyoto University qiang@i.kyoto-u.ac.jp Abstract Based on an exponentially increasing number of academic articles, discovering and citing com- prehensive and appropriate resources has become a non-trivial task. Conventional citation rec- ommender methods suffer from severe information loss. For example, they do not consider the section of the paper that the user is writing and for which they need to find a citation, the re- latedness between the words in the local context (the text span that describes a citation), or the importance on each word from the local context. These shortcomings make such methods insuf- ficient for recommending adequate citations to academic manuscripts. In this study, we propose a novel embedding-based neural network called “dual attention model for citation recommendation (DACR)” to recommend citations during manuscript preparation. Our method adapts embedding of three semantic information: words in the local context, structural contexts 1, and the section on which a user is working. A neural network model is designed to maximize the similarity between the embedding of the three input (local context words, section and structural contexts) and the target citation appearing in the context. The core of the neural network model is composed of self-attention and additive attention, where the former aims to capture the relatedness between the contextual words and structural context, and the latter aims to learn the importance of them. The experiments on real-world datasets demonstrate the effectiveness of the proposed approach. 1 Introduction When writing an academic paper, one of the most frequent questions considered is: “Which paper should I cite at this place?” Based on the massive number of papers being published, it is impossible for a researcher to read every article that might be relevant to their study. Thus, recommending a handful of useful citations based on the contents of a working draft can significantly alleviate the burden of writing a paper. An example of the application scenario is demonstrated in Figure 1. Currently, many scholars rely on “keyword searches” on search engines, such as Google Scholar 2 and DBLP 3. However, keyword-based systems often generate unsatisfying results, because query words may not convey adequate information to reflect the context that needs to be supported (Jia and Saule, 2017; Jia and Saule, 2018). Researchers in various fields have proposed various methods to solve this problem. For example, studies in (McNee et al., 2002; Gori and Pucci, 2006; Caragea et al., 2013; K¨uc¸¨uktunc¸ et al., 2013; Jia and Saule, 2018) considered recommendations based on a collection of seed papers, and (Alzoghbi et al., 2015; Li et al., 2018) proposed methods using meta-data, such as authorship information, titles, abstracts, keyword lists, and publication years. However, when applying such methods to real-world paper-writing tasks, there is a lack of consideration for the local context of a citation within a draft, which can potentially lead to suboptimal results. Context-based recommendations adopt a more practical concept that generates potential …

Content truncated to 500 words