Towards Two-step Multi-document Summarisation for Evidence Based Medicine: A Quantitative Analysis

Towards Two-step Multi-document Summarisation for Evidence Based Medicine: A Quantitative Analysis Abeed Sarker , Diego Moll´a-Aliod Centre for Language Technology Macquarie University Sydney, NSW 2109 {abeed.sarker, diego.molla-aliod}@mq.edu.au C´ecile Paris CSIRO – ICT Centre Sydney, NSW 2122 cecile.paris@csiro.au Abstract We perform a quantitative analysis of data in a corpus that specialises on summarisation for Ev- idence Based Medicine (EBM). The intent of the analysis is to discover possible directions for performing automatic evidence-based sum- marisation. Our analysis attempts to ascertain the extent to which good, evidence-based, multi- document summaries can be obtained from indi- vidual single-document summaries of the source texts. We define a set of scores, which we call coverage scores, to estimate the degree of in- formation overlap between the multi-document summaries and source texts of various granu- larities. Based on our analysis, using several variants of the coverage scores, and the results of a simple task oriented evaluation, we ar- gue that approaches for the automatic generation of evidence-based, bottom-line, multi-document summaries may benefit by utilising a two-step approach: in the first step, content-rich, single- document, query-focused summaries are gener- ated; followed by a step to synthesise the infor- mation from the individual summaries. 1 Introduction Automatic summarisation is the process of pre- senting the important information contained in a source text in a compressed format. Such approaches have important applications in do- mains where lexical resources are abundant and users face the problem of information overload. One such domain is the medical domain, with the largest online database (PubMed1) contain- ing over 21 million published medical articles. Thus, a standard clinical query on this database returns numerous results, which are extremely time-consuming to read and analyse manually. This is a major obstacle to the practice of Ev- idence Based Medicine (EBM), which requires practitioners to refer to relevant published med- ical research when attempting to answer clinical 1http://www.ncbi.nlm.nih.gov/pubmed queries. Research has shown that practitioners re- quire botom-line evidence-based answers at point of care, but often fail to obtain them because of time constraints (Ely et al., 1999). 1.1 Motivation Due to the problems associated with the prac- tise of EBM, there is a strong motivation for au- tomatic summarisation/question-answering (QA) systems that can aid practitioners. While auto- matic text summarisation research in other do- mains (e.g., news) has made significant advances, research in the medical domain is still at an early stage. This can be attributed to various factors: (i) the process of answer generation for EBM re- quires practitioners to combine their own exper- tise with medical evidence, and automatic systems are only capable of summarising content present in the source texts; (ii) the medical domain is very complex with a large number of domain spe- cific terminologies and relationships between the terms that systems need to take into account when performing summarisation; and (iii) while there is an abundance of medical documents available electronically, specialised corpora for performing summarisation research in this domain are scarce. 1.2 Contribution We study a corpus that specialises on the …

Content truncated to 500 words