Evaluation Criteria
Related Papers
*SEM 2012 Shared Task: Resolving the Scope and Focus of Negation
The Joint Conference on Lexical and Compu- tational Semantics (*SEM) each year hosts a shared task on semantic related topics. In its first edition held in 2012, the shared task …...
CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes
The CoNLL-2012 shared task involved pre- dicting coreference in English, Chinese, and Arabic, using the final version, v5.0, of the OntoNotes corpus. It was a follow-on to the English-only task …...
Generating Coherent Event Schemas at Scale
Chambers and Jurafsky (2009) demonstrated that event schemas can be automatically in- duced from text corpora. However, our analy- sis of their schemas identifies several weak- nesses, e.g., some schemas …...
Building the Directed Semantic Graph for Coherent Long Text Generation
AbstractGenerating long text conditionally depending on the short input text has recently attracted more and more research efforts. Most existing approaches focus more on introducing extra knowledge to supplement the …...
Perturbation CheckLists for Evaluating NLG Evaluation Metrics
AbstractNatural Language Generation (NLG) evaluation is a multifaceted task requiring assessment of multiple desirable criteria, e.g., fluency, coherency, coverage, relevance, adequacy, overall quality, etc. Across existing datasets for 6 NLG …...
Rogue Scores
AbstractCorrect, comparable, and reproducible model evaluation is essential for progress in machine learning. Over twenty years, thousands of language and vision models have been evaluated with a popular metric called …...