Эталонға негізделген метрикалар
Related Papers
There’s No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction
Current methods for automatically evaluating grammatical error correction (GEC) systems rely on gold-standard references. However, these methods suffer from penalizing gram- matical edits that are correct but not in the …...
An Anchor-Based Automatic Evaluation Metric for Document Summarization
AbstractThe widespread adoption of reference-based automatic evaluation metrics such as ROUGE has promoted the development of document summarization. In this paper, we consider a new protocol for designing reference-based metrics …...
Diverse and Relevant Visual Storytelling with Scene Graph Embeddings
AbstractA problem in automatically generated stories for image sequences is that they use overly generic vocabulary and phrase structure and fail to match the distributional characteristics of human-generated text. We …...
Generating Negative Samples by Manipulating Golden Responses for Unsupervised Learning of a Response Evaluation Model
AbstractEvaluating the quality of responses generated by open-domain conversation systems is a challenging task. This is partly because there can be multiple appropriate responses to a given dialogue history. Reference-based …...
REAM♯: An Enhancement Approach to Reference-based Evaluation Metrics for Open-domain Dialog Generation
The lack of reliable automatic evaluation met- rics is a major impediment to the development of open-domain dialogue systems. Various reference-based metrics have been proposed to calculate a score between …...