Human Assessment
Related Papers
Crowd-based MT Evaluation for non-English Target Languages
This paper investigates the feasibility of using crowd-sourcing services for the hu- man assessment of machine translation quality of translations into non-English tar- get languages. Non-expert graders are hired through …...
Human annotation of ASR error regions: Is “gravity” a sharable concept for human annotators?
AbstractThis paper is concerned with human assessments of the severity of errors in ASR outputs. We did not design any guidelines so that each annotator involved in the study could …...
Accurate Evaluation of Segment-level Machine Translation Metrics
Evaluation of segment-level machine transla- tion metrics is currently hampered by: (1) low inter-annotator agreement levels in human as- sessments; (2) lack of an effective mechanism for evaluation of translations …...
What does this Emoji Mean? A Vector Space Skip-Gram Model for Twitter Emojis
AbstractEmojis allow us to describe objects, situations and even feelings with small images, providing a visual and quick way to communicate. In this paper, we analyse emojis used in Twitter …...
Aspect-based summarization of pros and cons in unstructured product reviews
AbstractWe developed three systems for generating pros and cons summaries of product reviews. Automating this task eases the writing of product reviews, and offers readers quick access to the most …...