Towards assessing depth of argumentation

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3308–3317, Osaka, Japan, December 11-17 2016. Towards assessing depth of argumentation Manfred Stede Applied Computational Linguistics UFS Cognitive Sciences University of Potsdam / Germany stede@uni-potsdam.de Abstract For analyzing argumentative text, we propose to study the ‘depth’ of argumentation as one im- portant component, which we distinguish from argument quality. In a pilot study with German newspaper commentary texts, we asked students to rate the degree of argumentativeness, and then looked for correlations with features of the annotated argumentation structure and the rhetorical structure (in terms of RST). The results indicate that the human judgements correlate with our operationalization of depth and with certain structural features of RST trees. 1 Introduction In recent years, argumentation mining has emerged as a discipline that aims to identify portions of argumentation in text and moreover – in some of the work – to relate them to one another, so that the full structure of arguments is being represented. A natural application for this new field is the mining of arguments in customer reviews (e.g., (Villalba and Saint-Dizier, 2012)), where the goal is to move beyond finding positive and negative statements by also finding the reasons that customers provide for their judgements. But the general task can be applied in many other domains as well. For example, the early research of Mochales Palau and Moens (2009) sought to identify premises and conclusions in legal text; another early work on finding theses in student essays (Burstein et al., 2003) has recently been extended to detecting more argumentative structure in such essays (Stab and Gurevych, 2014), (Nguyen and Litman, 2015); other genres that are being tackled include online dialogue (Oraby et al., 2015), multi-party dialogue (Budzynska et al., 2013) and scientific papers (Kirschner et al., 2015). Despite the manifold recent activities, the field is still young, and a number of basic issues still require attention. This concerns the design of annotation schemes and possibly finding a consensus on them, but also the more fundamental problem of defining the notion of, and identifying instances of, argumentative text. While it is tempting to assume that earlier work on text-level subjectivity classification can be used to determine whether some (portion of) text is argumentative, we argue below that this is generally not the case. Going further, we posit that ‘argumentativeness’ is a matter of degree, and that determining this degree should also be considered a subtask of argumentation mining. We use the genre of newspaper editorials to conduct a small study where human raters are asked to assess how ‘argumentative’ different texts are, or in other words, what their ‘argumentative depth’ is. We will defend the introduction of this term by demonstrating that depth is to be distinguished from argument quality, which is already an object of study in the community. Our next task then is to explain how such judgements arise, i.e., to find features that differentiate texts of different argumentative depths. We show that …

Content truncated to 500 words