Vox Populi Annotation: Measuring Intensity of Ideological Perspectives by Aggregating Group Judgments
10017Authors
nan
Abstract
AbstractPolarizing discussions about political and social issues are common in mass media. Annotations on the degree to which a sentence expresses an ideological perspective can be valuable for evaluating computer programs that can automatically identify strongly biased sentences, but such annotations remain scarce. We annotated the intensity of ideological perspectives expressed in 250 sentences by aggregating judgments from 18 annotators. We proposed methods of determining the number of annotators and assessing reliability, and showed the annotations were highly consistent across different annotator groups.
Keywords
Full Content
nan
Labeled Text
AbstractPolarizing discussions about political and social issues are common in mass media. Annotations on the degree to which a sentence expresses an ideological perspective can be valuable for evaluating computer programs that can automatically identify strongly biased sentences, but such annotations remain scarce. We annotated the intensity of ideological perspectives expressed in 250 sentences by aggregating judgments from 18 annotators. We proposed methods of determining the number of annotators and assessing reliability, and showed the annotations were highly consistent across different annotator groups.
Publication Details
- Paper ID:
- 10017
- Year:
- 2008
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan