A Multi-Layered Annotated Corpus of Scientific Papers

A Multi-Layered Annotated Corpus of Scientific Papers Beatriz Fisas, Francesco Ronzano, Horacio Saggion DTIC - TALN Research Group, Pompeu Fabra University c/Tanger 122, 08018 Barcelona, Spain {beatriz.fisas, francesco.ronzano, horacio.saggion}@upf.edu Abstract Scientific literature records the research process with a standardized structure and provides the clues to track the progress in a scientific field. Understanding its internal structure and content is of paramount importance for natural language processing (NLP) technologies. To meet this requirement, we have developed a multi-layered annotated corpus of scientific papers in the domain of Computer Graphics. Sentences are annotated with respect to their role in the argumentative structure of the discourse. The purpose of each citation is specified. Special features of the scientific discourse such as advantages and disadvantages are identified. In addition, a grade is allocated to each sentence according to its relevance for being included in a summary.To the best of our knowledge, this complex, multi-layered collection of annotations and metadata characterizing a set of research papers had never been grouped together before in one corpus and therefore constitutes a newer, richer resource with respect to those currently available in the field. Keywords: multi-layered annotated corpus, scientific discourse, citations, summarization gold standard 1. Introduction The development of natural language processing tools for information extraction or document summarization tailored to scientific literature will provide quick tracking of scientific creativity and innovation. Easy access to challenges faced by the researchers, their results and contributions, and how these relate to the works of other researchers highlighting the novelties or advantages of the explored scientific project may inspire new approaches in a line of investigation. With the aim of supporting automated analysis and thus easier access to this information, we have generated a multi-layered annotated corpus of scientific discourse. In this article, after introducing our multi-layered scienfic annotation schema, we describe the way we collaboratively annotate our corpus so as to create its gold standard version. Corpus annotations have been provided in two stages. In the first stage (Fisas et al., 2015), annotators were asked to characterize the argumentative structure of papers by associating each sentence to one over 5 categories (Challenge, Background, Approach, Outcome and Future Work) and eventually specifying for Challenge sentences a subcategory among Goal and Hypothesis and distingushing among the Outcome sentences the ones that describe an authors’ Contribution. Based on the work of Liakata et al. (2010) and Teufel (2010) we developed an annotation schema and produced an annotated corpus. Its quality was evaluated in terms of the inter-annotator agreement (K=0.66) comparable to the values attained by the afore- mentioned researchers. The results were analysed by category and 5 main areas were identified in the articles where the middle 50% of sen- tences included in each category were located. The output was a gold standard annotated corpus (10,777 sentences; 40 documents) in the domain of Computer Graphics. This corpus constitutes a valid dataset to experiment with automatic sentence classification algorithms. In the second phase of the annotation, a step further into some other aspects of …

Content truncated to 500 words