Comparative Evaluation
Related Papers
A Common Framework for Syntactic Annotation
It is widely recognized that the proliferation of annotation schemes runs counter to the need to re-use language resources, and that standards for linguistic annotation are becoming increasingly mandatory. To …...
Evaluation Tool for Rule-based Anaphora Resolution Methods
In this paper we argue that comparative evaluation in anaphora resolution has to be performed using the same pre-processing tools and on the same set of data. The paper proposes …...
Incremental Dialogue Processing in a Micro-Domain
This paper describes a fully incremental dia- logue system that can engage in dialogues in a simple domain, number dictation. Be- cause it uses incremental speech recognition and prosodic analysis, …...
On the practice of error analysis for machine translation evaluation
AbstractError analysis is a means to assess machine translation output in qualitative terms, which can be used as a basis for the generation of error profiles for different systems. As …...
A Comparative Evaluation Methodology for NLG in Interactive Systems
AbstractInteractive systems have become an increasingly important type of application for deployment of NLG technology over recent years. At present, we do not yet have commonly agreed terminology or methodology …...
A method for in-depth comparative evaluation: How (dis)similar are outputs of pos taggers, dependency parsers and coreference resolvers really?
AbstractThis paper proposes a generic method for the comparative evaluation of system outputs. The approach is able to quantify the pairwise differences between two outputs and to unravel in detail …...
Duluth at SemEval-2020 Task 12: Offensive Tweet Identification in English with Logistic Regression
AbstractThis paper describes the Duluth systems that participated in SemEval–2020 Task 12, Multilingual Offensive Language Identification in Social Media (OffensEval–2020). We participated in the three English language tasks. Our systems …...