Summarization Systems
Related Papers
The Next Step for Multi-Document Summarization: A Heterogeneous Multi-Genre Corpus Built with a Novel Construction Approach
AbstractResearch in multi-document summarization has focused on newswire corpora since the early beginnings. However, the newswire genre provides genre-specific features such as sentence position which are easy to exploit in …...
Studying Summarization Evaluation Metrics in the Appropriate Scoring Range
AbstractIn summarization, automatic evaluation metrics are usually compared based on their ability to correlate with human judgments. Unfortunately, the few existing human judgment datasets have been created as by-products of …...
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems
AbstractNeural network-based models augmented with unsupervised pre-trained knowledge have achieved impressive performance on text summarization. However, most existing evaluation methods are limited to an in-domain setting, where summarizers are trained …...
How well do you know your summarization datasets?
State-of-the-art summarization systems are trained and evaluated on massive datasets scraped from the web. Despite their preva- lence, we know very little about the underly- ing characteristics (data noise, summarization …...
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
AbstractHuman evaluation is the foundation upon which the evaluation of both summarization systems and automatic metrics rests. However, existing human evaluation studies for summarization either exhibit a low inter-annotator agreement …...