Pyramid-based Summary Evaluation Using Abstract Meaning Representation
20378Authors
nan
Abstract
AbstractWe propose a novel metric for evaluating summary content coverage. The evaluation framework follows the Pyramid approach to measure how many summarization content units, considered important by human annotators, are contained in an automatic summary. Our approach automatizes the evaluation process, which does not need any manual intervention on the evaluated summary side. Our approach compares abstract meaning representations of each content unit mention and each summary sentence. We found that the proposed metric complements well the widely-used ROUGE metrics.
Keywords
Full Content
nan
Labeled Text
AbstractWe propose a novel metric for evaluating summary content coverage. The evaluation framework follows the Pyramid approach to measure how many summarization content units, considered important by human annotators, are contained in an automatic summary. Our approach automatizes the evaluation process, which does not need any manual intervention on the evaluated summary side. Our approach compares abstract meaning representations of each content unit mention and each summary sentence. We found that the proposed metric complements well the widely-used ROUGE metrics.
Publication Details
- Paper ID:
- 20378
- Year:
- 2017
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan