A Shared Task on Argumentation Mining in Newspaper Editorials

Proceedings of the 2nd Workshop on Argumentation Mining, pages 35–38, Denver, Colorado, June 4, 2015. c⃝2015 Association for Computational Linguistics A Shared Task on Argumentation Mining in Newspaper Editorials Johannes Kiesel Khalid Al-Khatib Matthias Hagen Benno Stein Bauhaus-Universit¨at Weimar 99421 Weimar, Germany <first name>.<last name>@uni-weimar.de Abstract This paper proposes a shared task for the identification of the argumentative structure in newspaper editorials. By the term “argumen- tative structure” we refer to the sequence of ar- gumentative units in the text along with the re- lations between them. The main contribution is a large-scale dataset with more than 200 an- notated editorials, which shall help argumen- tation mining researchers to evaluate and com- pare their systems in a standardized manner. The paper details how we model and manu- ally identify argumentative structures in order to build this evaluation resource. Altogether, we consider the proposed task as a construc- tive step towards improving writing assistance systems and debating technologies. 1 Introduction Even though argumentation theories have been stud- ied extensively in many areas (e.g., philosophy), us- ing these theories for mining real world text is a rela- tively new direction of research. Recently, argumen- tation mining has attracted many Natural Language Processing (NLP) researchers with papers published in major conferences and even a specialized work- shop series. Argumentation mining typically refers to the tasks of automatically identifying the argumentative units of a given text and the relations between them (i.e., support or attack). The automatic analysis of this discourse structure has several applications, such as supporting writing skills or assisting information- seeking users in constructing a solid personal stand- point on controversial topics. To further foster the young field of argumenta- tion mining, we propose a respective shared task to evaluate the current state of the art and compare to newly emerging ideas. According to the standard understanding of argumentation mining, we propose two focused sub-tasks: (1) unit identification and (2) relation extraction. The shared task will allow researchers to evaluate their systems in an open but standardized competition, which will help to push forward argumentation mining research. For the corpus of the shared task, we are cur- rently annotating a collection of newspaper editori- als. These articles convey the opinions of their au- thors towards specific topics and try to persuade the readers of theses opinions. In order to do so, the au- thors support their opinions by reasons, which leads to an argumentative discourse. We plan to anno- tate at least 200 editorials from three different on- line newspapers paragraph-wise. Participants can use about two thirds of this corpus for training while the remainder will be used for evaluation. The paper is structured as follows. Section 2 de- scribes the argumentative structure that participants have to extract from the corpus that is described in detail in Section 3. Section 4 proposes the general scheme of the two sub-tasks while the task submis- sion is outlined in Section 5. Conclusions are given in Section 6. 2 Argumentation Model As the basis …

Content truncated to 500 words