Analyzing Citation-Distance Networks for Evaluating Publication Impact

Analyzing Citation-Distance Networks for Evaluating Publication Impact Drahomira Herrmannova, Petr Knoth, Robert Patton Oak Ridge National Laboratory, The Open University, Oak Ridge National Laboratory Oak Ridge, TN, USA; Milton Keynes, UK; Oak Ridge, TN, USA herrmannovad@ornl.gov, petr.knoth@open.ac.uk, pattonrm@ornl.gov Abstract Studying citation patterns of scholarly articles has been of interest to many researchers from various disciplines. While the relationship of citations and scientific impact has been widely studied in the literature, in this paper we develop the idea of analyzing the semantic distance of scholarly articles in a citation network (citation-distance network) to uncover patterns that reflect scientific impact. More specifically, we compare two types of publications in terms of their citation-distance patterns, seminal publications and literature reviews, and focus on their referencing patterns as well as on publications which cite them. We show that seminal publications are associated with a larger semantic distance, measured using the content of the articles, between their references and the citing publications, while literature reviews tend to cite publications from a wider range of topics. Our motivation is to understand and utilize this information to create new research evaluation metrics which would better reflect scientific impact. Keywords: citation networks, publication impact, semantic similarity Acknowledgments This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government re- tains and the publisher, by accepting the article for pub- lication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Gov- ernment purposes. The Department of Energy will pro- vide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan1. 1. Introduction With the enormous and ever-growing number of research articles being published every year (Jinha, 2010; Laakso and Bj¨ork, 2012), researchers more and more often resort to using research evaluation metrics such as journal and publication citation indexes, as a proxy to quality and im- portance. Research metrics have been applied in various scenarios, from search and recommendation (Carevic and Schaer, 2014; Belter, 2016), to grant and tenure awards (Meho, 2007). Due to many drawbacks and limitations of the purely citation-based methods (Seglen, 1992; Seglen, 1997; Priem et al., 2010), recent years have seen the emer- gence of many new approaches and alternatives to the tra- ditional bibliometrics, most notably metrics often referred to collectively as altmetrics (Priem et al., 2010) and webo- metrics (Almind and Ingwersen, 1997), which rely on data collected from the Web, such as download counts and so- cial and news media mentions, and semantometrics (Knoth and Herrmannova, 2014), which measure how far each dis- covery takes us by utilizing publication full texts. While citation networks have been widely studied in the literature, a number of works have recently developed the idea of studying citation patterns in combination with con- tent similarity. In this paper we further explore this area 1http://energy.gov/downloads/ doe-public-access-plan and …

Content truncated to 500 words