metric ID: kappa

Related Papers

Iterative Refinement and Quality Checking of Annotation Guidelines — How to Deal Effectively with Semantically Sloppy Named Entity Types, such as Pathological Phenomena

Authors: nan

Publication: TIPSTER (2012)

DOI: nan

AbstractWe here discuss a methodology for dealing with the annotation of semantically hard to delineate, i.e., sloppy, named entity types. To illustrate sloppiness of entities, we treat an example from …...

13565

The Problem with Kappa

Authors: nan

Publication: TIPSTER (2012)

DOI: nan

It is becoming clear that traditional evaluation measures used in Computational Linguistics (including Error Rates, Accuracy, Recall, Precision and F-measure) are of limited value for unbiased evaluation of systems, and …...

43692

Getting Reliable Annotations for Sarcasm in Online Dialogues

Authors: nan

Publication: TIPSTER (2014)

DOI: nan

AbstractThe language used in online forums differs in many ways from that of traditional language resources such as news. One difference is the use and frequency of nonliteral, subjective dialogue …...

17076

Emotion Corpus Construction Based on Selection from Hashtags

Authors: nan

Publication: TIPSTER (2016)

DOI: nan

AbstractThe availability of labelled corpus is of great importance for supervised learning in emotion classification tasks. Because it is time-consuming to manually label text, hashtags have been used as naturally …...

16194

Reproducibility and Automation of the Appraisal Taxonomy

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

AbstractThere is a lack of reproducibility in results from experiments that apply the Appraisal taxonomy. Appraisal is widely used by linguists to study how people judge things or people. Automating …...

4888