Annotating Arguments in a Corpus of Opinion Articles

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1890–1899 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1890 Annotating Arguments in a Corpus of Opinion Articles Gil Rocha †, Lu´ıs Trigo †, Henrique Lopes Cardoso †, Rui Sousa-Silva ⋆, Paula Carvalho ‡, Bruno Martins ‡, Miguel Won ‡ † LIACC / Faculdade de Engenharia da Universidade do Porto, Porto, Portugal, ⋆ CLUP / Faculdade de Letras da Universidade do Porto, Porto, Portugal, ‡ INESC-ID, Lisboa, Portugal {gil.rocha, ltrigo, hlc}@fe.up.pt, rssilva@letras.up.pt, pcc@inesc-id.pt, {bruno.g.martins, miguelwon}@tecnico.ulisboa.pt Abstract Interest in argument mining has resulted in an increasing number of argument annotated corpora. However, most focus on English texts with explicit argumentative discourse markers, such as persuasive essays or legal documents. Conversely, we report on the first extensive and consolidated Portuguese argument annotation project focused on opinion articles. We briefly describe the annotation guidelines based on a multi-layered process and analyze the manual annotations produced, highlighting the main challenges of this textual genre. We then conduct a comprehensive inter-annotator agreement analysis, including argumentative discourse units, their classes and relations, and resulting graphs. This analysis reveals that each of these aspects tackles very different kinds of challenges. We observe differences in annotator profiles, motivating our aim of producing a non-aggregated corpus containing the insights of every annotator. We note that the interpretation and identification of token-level arguments is challenging; nevertheless, tasks that focus on higher-level components of the argument structure can obtain considerable agreement. We lay down perspectives on corpus usage, exploiting its multi-faceted nature. Keywords: Argument annotation, Inter-annotator agreement, Opinion articles 1. Introduction Argumentation is the process of exposing and justify- ing one’s points of view, with the aim of conveying a logical reasoning through a set of semantically related propositions. In written form, such reasoning may be more or less explicit, depending on the text genre, on the author’s writing style and argumentation strategy, among others. The process of annotating arguments in text starts by identifying argumentative discourse units (ADUs) – such as premises and conclusions. These can then be used to build argument diagrams, in which re- lations between ADUs become explicit. From a the- oretical point of view, several argumentation models have been proposed (van Eemeren et al., 2014), aim- ing at explaining the ways in which human language is used in argumentation processes. Most argument anno- tation projects focus on English, and have used varied argumentation models, in particular based on work by Freeman (2011), Toulmin (1958), and Walton (1996). This paper reports the results of an argument annotation project that has sought to explore a hybrid annotation model, taking advantage of the best practices identified in previous studies, while at the same time fostering the potential use of the resulting annotated corpus for new purposes. These include not only argument min- ing (Stede and Schneider, 2019), but also other related tasks such as proposition classification and sentiment analysis. The corpus consists of opinion articles writ- ten in Portuguese. Opinion articles …

Content truncated to 500 words