Annotation Efficiency
metric
ID: annotationefficiency
Related Papers
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants
AbstractIn Dynamic Adversarial Data Collection (DADC), human annotators are tasked with finding examples that models struggle to predict correctly. Models trained on DADC-collected training data have been shown to be …...
Quality and Efficiency of Manual Annotation: Pre-annotation Bias
AbstractThis paper presents an analysis of annotation using an automatic pre-annotation for a mid-level annotation complexity task - dependency syntax annotation. It compares the annotation efforts made by annotators using …...