Aspect-based Document Similarity for Research Papers

Proceedings of the 28th International Conference on Computational Linguistics, pages 6194–6206 Barcelona, Spain (Online), December 8-13, 2020 6194 Aspect-based Document Similarity for Research Papers Malte Ostendorff1,2 Terry Ruas3 Till Blume4 Bela Gipp2,3 Georg Rehm1 1German Research Center for Artificial Intelligence, Berlin, Germany 2University of Konstanz, Konstanz, Germany 3University of Wuppertal, Wuppertal, Germany 4University of Kiel, Kiel, Germany malte.ostendorff@dfki.de Abstract Traditional document similarity measures provide a coarse-grained distinction between similar and dissimilar documents. Typically, they do not consider in what aspects two documents are similar. This limits the granularity of applications like recommender systems that rely on doc- ument similarity. In this paper, we extend similarity with aspect information by performing a pairwise document classification task. We evaluate our aspect-based document similarity ap- proach for research papers. Paper citations indicate the aspect-based similarity, i. e., the title of a section in which a citation occurs acts as a label for the pair of citing and cited paper. We apply a series of Transformer models such as RoBERTa, ELECTRA, XLNet, and BERT variations and compare them to an LSTM baseline. We perform our experiments on two newly constructed datasets of 172,073 research paper pairs from the ACL Anthology and CORD-19 corpus. Ac- cording to our results, SciBERT is the best performing system with F1-scores of up to 0.83. A qualitative analysis validates our quantitative results and indicates that aspect-based document similarity indeed leads to more fine-grained recommendations. 1 Introduction Recommender systems assist researchers in finding relevant papers for their work. When user feed- back is sparse or unavailable, content-based approaches and corresponding document similarity mea- sures are employed (Beel et al., 2016). Recommender systems present a candidate document depending on whether it is similar or dissimilar to the seed document. This coarse-grained similarity assessment (similar or not) neglects the many facets that can make two documents similar. Concerning the general concept of similarity, Goodman (1972), and B¨ar et al. (2011) even argue that similarity is an ill-defined notion unless one can say to what aspects the similarity relates. In recommender systems for scientific papers, the similarity is often concerned with multiple facets of the presented research, e. g., method, findings (Huang et al., 2020). Given that document similarity can differentiate research aspects, one could obtain tailored recommendations. For instance, a paper with similar methods but different findings could be recommended. Such a recommender system would facilitate the discovery of analogies in re- search literature (Chan et al., 2018). We describe the underlying multiple aspect similarity in research papers as aspect-based document similarity. Figure 1 contrasts aspect-based with aspect-free similarity (traditional). Following the research paper example, aspect a1 concerns findings and aspect a2 methods (red and green in Figure 1b). In prior work (Ostendorff et al., 2020), we propose to infer an aspect for document similarity for- mulating the problem as a multi-class classification of document pairs. We extend our prior work to a multi-label scenario and focus on scientific literature instead of Wikipedia articles. Similar to the work of Jiang et al. …

Content truncated to 500 words