Crowdsourcing as a preprocessing for complex semantic annotation tasks
16488Authors
nan
Abstract
AbstractThis article outlines a methodology that uses crowdsourcing to reduce the workload of experts for complex semantic tasks. We split turker-annotated datasets into a high-agreement block, which is not modified, and a low-agreement block, which is re-annotated by experts. The resulting annotations have higher observed agreement. We identify different biases in the annotation for both turkers and experts.
Keywords
Full Content
nan
Labeled Text
AbstractThis article outlines a methodology that uses crowdsourcing to reduce the workload of experts for complex semantic tasks. We split turker-annotated datasets into a high-agreement block, which is not modified, and a low-agreement block, which is re-annotated by experts. The resulting annotations have higher observed agreement. We identify different biases in the annotation for both turkers and experts.
Publication Details
- Paper ID:
- 16488
- Year:
- 2014
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan