Measuring the Evolution of a Scientific Field through Citation Frames

Measuring the Evolution of a Scientific Field through Citation Frames David Jurgens University of Michigan jurgens@umich.edu Srijan Kumar Stanford University srijan@stanford.edu Raine Hoover Stanford University raine@stanford.edu Dan McFarland Stanford University dmcfarla@stanford.edu Dan Jurafsky Stanford University jurafsky@stanford.edu Abstract Citations have long been used to character- ize the state of a scientific field and to iden- tify influential works. However, writers use citations for different purposes, and this var- ied purpose influences uptake by future schol- ars. Unfortunately, our understanding of how scholars use and frame citations has been lim- ited to small-scale manual citation analysis of individual papers. We perform the largest be- havioral study of citations to date, analyzing how scientific works frame their contributions through different types of citations and how this framing affects the field as a whole. We introduce a new dataset of nearly 2,000 cita- tions annotated for their function, and use it to develop a state-of-the-art classifier and la- bel the papers of an entire field: Natural Lan- guage Processing. We then show how differ- ences in framing affect scientific uptake and reveal the evolution of the publication venues and the field as a whole. We demonstrate that authors are sensitive to discourse structure and publication venue when citing, and that how a paper frames its work through citations is pre- dictive of the citation count it will receive. Fi- nally, we use changes in citation framing to show that the field of NLP is undergoing a sig- nificant increase in consensus. 1 Introduction Authors use citations to frame their contributions and connect to an intellectual lineage (Latour, 1987). An author’s scientific frame employs citations in multiple ways (Figure 1) so as to build a strong Unlike CITE, we use the method of CITE, which has been used previously for parsing (CITE). Contrast Use Background Figure 1: Examples of citation functionality. and multifaceted argument. These differences in ci- tations have been examined extensively within the context of a single paper (Swales, 1986; White, 2004; Ding et al., 2014). However, we know rela- tively little about how these citation frames develop over time within a field and what impact they have on scientific uptake. Answering these questions has been largely hin- dered by the lack of a dataset showing how citations function at the field scale. Here, we perform the first field-scale study of citation framing by first de- veloping a state-of-the-art method for automatically classifying citation function and then applying this method to an entire field’s literature to quantify the effects and evolution of framing. Analyzing large-scale changes in citation fram- ing requires an accurate method for classifying the function a citation plays towards furthering an ar- gument. Due to the difficulty of interpreting ci- tation intent, many prior works performed manual analysis (Moravcsik and Murugesan, 1975; Swales, 1990; Harwood, 2009) and only recently have au- tomated approaches been developed (Teufel et al., 2006b; Valenzuela et al., 2015). Here, we unify core aspects of several prior citation annotation schemes (White, 2004; Ding et al., 2014; …

Content truncated to 500 words