IMHO Fine-Tuning Improves Claim Detection

Proceedings of NAACL-HLT 2019, pages 558–563 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 558 IMHO Fine-Tuning Improves Claim Detection Tuhin Chakrabarty Dept. Of Computer Science Columbia University tc2896@columbia.edu Christopher Hidey Dept. Of Computer Science Columbia University ch3085@columbia.edu Kathleen McKeown Dept. Of Computer Science Columbia University kathy@cs.columbia.edu Abstract Claims are the central component of an argu- ment. Detecting claims across different do- mains or data sets can often be challenging due to their varying conceptualization. We propose to alleviate this problem by fine tun- ing a language model using a Reddit corpus of 5.5 million opinionated claims. These claims are self-labeled by their authors using the in- ternet acronyms IMO/IMHO (in my (humble) opinion). Empirical results show that using this approach improves the state of art perfor- mance across four benchmark argumentation data sets by an average of 4 absolute F1 points in claim detection. As these data sets include diverse domains such as social media and stu- dent essays this improvement demonstrates the robustness of fine-tuning on this novel corpus. 1 Introduction Toulmin’s influential work on argumentation (2003) introduced a claim as an assertion that de- serves our attention. More recent work describes a claim as a statement that is in dispute and that we are trying to support with reasons (Govier, 2010). While some traits of claims are defined by their context, such as that claims usually need some support to make up a ’complete’ argument (e.g., premises, evidence, or justifications), the ex- act definition of a claim may vary depending on the domain, register, or task. Daxenberger et al. (2017) try to solve the problem of claim concep- tualization by training models across one data set and testing on others, but their cross-domain claim detection experiments mostly led to decreased re- sults over in-domain experiments. To demonstrate that some properties of claims are shared across domains, we create a diverse and rich corpus mined from Reddit and evaluate on held out datasets from different sources. We use Universal Language Model Fine-Tuning (ULM- FiT) (Howard and Ruder, 2018), which pre-trains a language model (LM) on a large general-domain corpus and fine-tunes it on our Reddit corpus be- fore training a final classifier to identify claims on various data sets. We make the following contributions: • We release a dataset of 5.5 million opinion- ated claims from Reddit,1 which we hope will be useful for computational argumentation. • We show transfer learning helps in the detec- tion of claims with varying definitions and conceptualizations across data sets from di- verse domains such as social media and stu- dent essays. • Empirical results show that using the Red- dit corpus for language model fine-tuning improves the state-of-the-art performance across four benchmark argumentation data sets by an average of 4 absolute F1 points in claim detection. 2 Related Work Argumentation mining (AM) is a research field within the growing area of computational argu- mentation. The tasks pursued within this field are highly challenging and include segmenting …

Content truncated to 500 words