Bridging Argument Quality and Deliberative Quality Annotations with Adapters

Findings of the Association for Computational Linguistics: EACL 2023, pages 2469–2488 May 2-6, 2023 ©2023 Association for Computational Linguistics Bridging Argument Quality and Deliberative Quality Annotations with Adapters Neele Falk and Gabriella Lapesa Institute for Natural Language Processing, University of Stuttgart {neele.falk,gabriella.lapesa}@ims.uni-stuttgart.de Abstract Assessing the quality of an argument is a com- plex, highly subjective task, influenced by het- erogeneous factors (e.g., prior beliefs of the an- notators, topic, domain, and application), and crucial for its impact in downstream tasks (e.g., argument retrieval or generation). Both the Argument Mining and the Social Science com- munity have devoted plenty of attention to it, resulting in a wide variety of argument qual- ity dimensions and a large number of anno- tated resources. This work aims at a better understanding of how the different aspects of argument quality relate to each other from a practical point of view. We employ adapter- fusion (Pfeiffer et al., 2021) as a multi-task learning framework which a) can improve the prediction of individual quality dimensions by injecting knowledge about related dimensions b) is efficient and modular and c) can serve as an analysis tool to investigate relations be- tween different dimensions. We conduct ex- periments on 6 datasets and 20 quality dimen- sions. We find that the majority of the dimen- sions can be learned as a weighted combination of other quality aspects, and that for 8 dimen- sions adapter fusion improves quality predic- tion. Last, we show the benefits of this ap- proach by improving the performance in an ex- trinsic, out-of-domain task: prediction of mod- erator interventions in a deliberative forum. 1 Introduction Although people have been dealing with the art of persuasion since ancient times, there are many answers to the question of what constitutes a good argument or good argumentation, and none can be considered the best: the quality of arguments is complex, subjective and depends on the context in which the quality is assessed and on the prerequi- sites (attitudes and values) of the one who judges it. Despite the high complexity of this task, a con- siderable research effort has been done to automat- ically model argument quality in different contexts (Wachsmuth et al., 2017a) due to its usefulness in downstream applications, such as automatic writ- ing assistants (Wambsganss et al., 2020), argument extraction (Alshomary et al., 2021) and generation (Gurcke et al., 2021). Social Science offers an- other whole field of theories and definitions about argument quality in which the focus is usually not only on the argument itself but on the discussion between participants thus emphasizing the deliber- ative goal of the discourse (Gerber et al., 2018). These two research communities, Argument Mining (AM) and Deliberative Theory (DT), have not only produced different theories of argument quality, but also a number of annotated datasets on the basis of which the models for the automatic assessment can be (or have been) trained. In both AM and DT, argument quality (AQ) and its Social Science counterpart, deliberative quality (DQ) are broken …

Content truncated to 500 words