Challenge or Empower: Revisiting Argumentation Quality in a News Editorial Corpus

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 454–464 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 454 Challenge or Empower: Revisiting Argumentation Quality in a News Editorial Corpus Roxanne El Baff Henning Wachsmuth ∗ Khalid Al-Khatib Benno Stein Bauhaus-Universität Weimar, Weimar, Germany, <first>{.<last>}+@uni-weimar.de ∗ Paderborn University, Paderborn, Germany, henningw@upb.de Abstract News editorials are said to shape public opin- ion, which makes them a powerful tool and an important source of political argumentation. However, rarely do editorials change anyone’s stance on an issue completely, nor do they tend to argue explicitly (but rather follow a subtle rhetorical strategy). So, what does argumen- tation quality mean for editorials then? We de- velop the notion that an effective editorial chal- lenges readers with opposing stance, and at the same time empowers the arguing skills of read- ers that share the editorial’s stance — or even challenges both sides. To study argumentation quality based on this notion, we introduce a new corpus with 1000 editorials from the New York Times, annotated for their perceived ef- fect along with the annotators’ political orien- tations. Analyzing the corpus, we find that an- notators with different orientation disagree on the effect significantly. While only 1% of all editorials changed anyone’s stance, more than 5% meet our notion. We conclude that our cor- pus serves as a suitable resource for studying the argumentation quality of news editorials. 1 Introduction A news editorial is an article that argues in favor of a particular stance on a usually timely controversial issue, such as the relocation of the US embassy in Israel to Jerusalem. Usually, it reflects the politi- cal ideology of the newspaper, aiming to persuade readers of the respective stance. Such editorials are said to have the power to shape the opinion of the masses. Similarly, they can increase or decrease the gap between readers with opposing beliefs (van Dijk, 1995). As such, news editorials represent an important resource for research on argument mining (Mochales and Moens, 2011) and debating technologies (Rinott et al., 2015). On the other hand, a single news editorial rarely changes the stance of a reader completely. More- over, many editorials do not put an explicit focus on arguments. Rather, they follow a subtle rhetori- cal strategy combining emotional anecdotes with hidden claims and ethotic evidence, among oth- ers (Al-Khatib et al., 2017). So, if not persuasive arguments, what makes a news editorial effective or ineffective then? In other words: How can we measure its argumentation quality? In this paper, we introduce a new corpus with 1000 news editorials from the New York Times where we consider argumentation quality from a dialectical perspective (van Eemeren and Grooten- dorst, 2004). While several quality dimensions are known in theory (Wachsmuth et al., 2017b), ex- isting approaches rely on subjective assessments of absolute (Persing and Ng, 2015) or relative (Habernal and Gurevych, 2016) persuasiveness. In contrast, our corpus captures quality in terms of whether an editorial brings …

Content truncated to 500 words