Tübingen at SemEval-2023 Task 4: What Can Stance Tell? A Computational Study on Detecting Human Values behind Arguments

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1763–1768 July 13-14, 2023 ©2023 Association for Computational Linguistics Tübingen at SemEval-2023 Task 4: What can Stance Tell? A Computational Study on Detecting Human Values behind Arguments Fidan Can Department of Linguistics, Eberhard Karls Universität Tübingen fidan.can@student.uni-tuebingen.de Abstract This paper describes the performance of a sys- tem which uses stance as an output instead of taking it as an input to identify 20 human values behind given arguments, based on two datasets for SemEval-2023 Task 4. The rationale was to draw a conclusion on whether predicting stance would help predict the given human values bet- ter. For this setup—predicting 21 labels—a pre-trained language model, RoBERTa-Large was used. The system had an F1-score of 0.50 for predicting these human values for the main test set while this score was 0.35 on the sec- ondary test set, and through further analysis, this paper aims to give insight into the problem of human value identification. 1 Introduction Arguing can take place in our everyday life to ex- press and rationalize ideas, and the way arguments are delivered can vary in their style, language and goal (Boltuži´c and Šnajder, 2014). In parallel with this variation, it is stated that it is a challenge to detect values expressed in the arguments as they are not always opinionated explicitly (Kiesel et al., 2022). Since argumentation is a part of our every- day life—for instance, while making decisions—it is likely that our values play a role, although this may not always be overt. Therefore, a large amount of literature on human value studies is an under- standable effort. With this study, I hope to con- tribute to computational approaches to identifying human values behind arguments. Kiesel et al. (2023) describe this current human value identification task as classifying whether a given textual argument falls into a given human value or not. Out of all the experiments, the versions with stance used as a separate label in addition to the existing 20 labels (resulting in 21 labels) outper- formed the rest. The underlying motivation was to mark whether the model would withdraw any infor- mation from the stance of the premise towards the conclusion. After the data pre-processing, the pre- trained transformer model, RoBERTa-Large (Liu et al., 2019), was implemented. This approach did not turn out to perform as well on the secondary test set, Nahj al-Balagha, as it did on the main test set when the F1-scores are taken into account—corresponding scores and ranks will be discussed later in the paper in addition to other approaches. This can very well ring a bell to how daunting it can be to identify human values due to their covert usage in arguments (Kiesel et al., 2022), especially considering that the arguments in the Nahj al-Balagha test set are from Islamic reli- gious texts with grandiloquent content that includes sermons (Mirzakhmedova et al., 2023). The aim of this paper is to present the role of stance and a …

Content truncated to 500 words