Augustine of Hippo at SemEval-2023 Task 4: An Explainable Knowledge Extraction Method to Identify Human Values in Arguments with SuperASKE

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1044–1053 July 13-14, 2023 ©2023 Association for Computational Linguistics Augustine Of Hippo at SemEval-2023 Task 4: An Explainable Knowledge Extraction Method to Identify Human Values in Arguments with SuperASKE Alfio Ferrara, Sergio Picascia, Elisabetta Rocchetti University of Milan, Via Celoria 18, 20133, Milan, Italy {alfio.ferrara, sergio.picascia, elisabetta.rocchetti}@unimi.it Abstract In this paper we present and discuss the results achieved by the "Augustine of Hippo" team at SemEval-2023 Task 4 about human value detection. In particular, we provide a quan- titative and qualitative reviews of the results obtained by SuperASKE, discussing respec- tively performance metrics and classification errors. Finally, we present our main contribu- tion: an explainable and unsupervised approach mapping arguments to concepts, followed by a supervised classification model mapping con- cepts to human values. 1 Introduction Argumentation is a communication act requiring a strong pragmatical component to be entirely under- stood by humans. This high-level language char- acterization encodes contextual information which, in turn, can be partitioned into linguistic, cognitive, physical, semantic, and social aspects. Depending on actual and perceived context, individuals adjust their communication in order to convey a message as effectively as possible: this leads to the possibil- ity of many concepts to remain implicit, especially those that can be deduced from context, which in- cludes people’s beliefs and values. SemEval-2023 Task 4 Kiesel et al. (2023a) aims at exploring novel methodologies for extracting human values from ar- gumentation texts written in English. Focusing on this type of contextual information has the benefit of improving argumentation analysis and genera- tion Kiesel et al. (2022). Our contribution as "Augustine of Hippo" team consists in SuperASKE1, an explainable system made of ASKE (Automated System for Knowledge Extraction) and Random Forest producing a binary output. Explanations are made possible by associat- ing human values to concepts extracted by ASKE, 1Docker container image available at sergiopicascia/ semeval-superaske which are ordered by Random Forest’s feature im- portance. Our model has a median performance with respect to other runs submitted for the task. The work is organized as follows: Section 2 pro- vides some background about the task of human values detection from arguments and the original ASKE model; Section 3 explains the behaviour of the whole framework; Section 4 specifies the details on how the experiment was run; Section 5 discusses the performances of the approach; Sec- tion 6 sums up the most important takings of the research. 2 Background SemEval-2023 Task 4 is about human value detec- tion. This task requires a classification model to recognise which among 20 values (or a subset of them) are present in a textual argument. Specif- ically, the subset of human values considered is taken from Kiesel et al. (2022) which, in turn, dis- cusses a selection of values inspired by Schwartz et al. (2012), Gouldner (1975), Brown and Crace (2002) and C. et al. (2020). The authors in Kiesel et al. (2022) describe 4 levels contributing to the value taxonomy, which can be …

Content truncated to 500 words