A Discourse-Driven Content Model for Summarising Scientific Articles Evaluated in a Complex Question Answering Task

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 747–757, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics A discourse-driven content model for summarising scientific articles evaluated in a complex question answering task Maria Liakata University of Warwick/ EMBL-EBI, UK M.Liakata@warwick.ac.uk Simon Dobnik University of Gothenburg, Sweden simon.dobnik@gu.se Shyamasree Saha EMBL-EBI, UK saha@ebi.ac.uk Colin Batchelor Royal Society of Chemistry, UK batchelorc@rsc.org Dietrich Rebholz-Schuhmann University of Zurich, Switzerland/ EMBL-EBI, UK rebholz@ebi.ac.uk Abstract We present a method which exploits auto- matically generated scientific discourse an- notations to create a content model for the summarisation of scientific articles. Full pa- pers are first automatically annotated using the CoreSC scheme, which captures 11 content- based concepts such as Hypothesis, Result, Conclusion etc at the sentence level. A content model which follows the sequence of CoreSC categories observed in abstracts is used to pro- vide the skeleton of the summary, making a distinction between dependent and indepen- dent categories. Summary creation is also guided by the distribution of CoreSC cate- gories found in the full articles, in order to adequately represent the article content. Fi- nally, we demonstrate the usefulness of the summaries by evaluating them in a complex question answering task. Results are very en- couraging as summaries of papers from auto- matically obtained CoreSCs enable experts to answer 66% of complex content-related ques- tions designed on the basis of paper abstracts. The questions were answered with a precision of 75%, where the upper bound for human summaries (abstracts) was 95%. 1 Introduction The publication boom of the last few years, espe- cially in the life sciences, has highlighted the need to facilitate automatic access to the information con- tent of articles. Researchers, curators, reviewers all need to process a continuously expanding flow of articles whether the purpose is to follow the state of the art, curate large knowledge bases or have a good working knowledge of their own and related disci- plines to assess progress in research. While a lot of effort has concentrated on information extraction of particular types of entities and relations from the scientific literature (Cohen and Hersh, 2005; Kim et al., 2009; Ananiadou et al., 2010; Kim et al., 2011), with a view to support scientists in obtain- ing relevant information from scientific articles and abstracts, less work has focussed on automatically combining such information in the form of a co- hesive summary which preserves the context. Re- searchers rely to a great extent on author-written ab- stracts, but the latter suffer from a number of prob- lems; they are less structured, vary significantly in terms of length, are often not self-contained and have been written independently of the main doc- ument (Teufel, 2010, p.83). Teufel (2001; 2010), (Teufel and Moens, 2002) identify argumentative zones within scientific arti- cles and use them to create use-targeted extractive summaries. Argumentative zones are annotations which designate the type of knowledge claim and rhetorical status for a sentence and how these relate to the communicative function …

Content truncated to 500 words