Rudolf Christoph Eucken at SemEval-2023 Task 4: An Ensemble Approach for Identifying Human Values from Arguments

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 660–663 July 13-14, 2023 ©2023 Association for Computational Linguistics Rudolf Christoph Eucken at SemEval-2023 Task 4: An Ensemble Approach for Identifying Human Values from Arguments Sougata Saha, Rohini Srihari State University of New York at Buffalo Department of Computer Science and Engineering {sougatas, rohini}@buffalo.edu Abstract The subtle human values we acquire through life experiences govern our thoughts and gets reflected in our speech. It plays an integral part in capturing the essence of our individu- ality and making it imperative to identify such values in computational systems that mimic hu- man actions. Computational argumentation is a field that deals with the argumentation capabili- ties of humans and can benefit from identifying such values. Motivated by that, we present an ensemble approach for detecting human val- ues from argument text. Our ensemble com- prises three models: (i) An entailment-based model for determining the human values based on their descriptions, (ii) A Roberta-based clas- sifier that predicts the set of human values from an argument. (iii) A Roberta-based classifier to predict a reduced set of human values from an argument. We experiment with different ways of combining the models and report our results. Furthermore, our best combination achieves an overall F1 score of 0.48 on the main test set. 1 Introduction Human values (Searle, 2003) play an integral role in determining how and why people react and respond to events in a particular way (Schwartz, 1994). We acquire them throughout our lifetime, and they largely govern our actions and manifest through our arguments. They play a significant role in defining our individuality and show a glimpse of our inner self by revealing our belief system. Motivated by why people reason and respond in a certain way, we present computational models ca- pable of automatically identifying such perceivable values from argument text. Given a set of 20 human value categories, the task (Kiesel et al., 2023) comprises classifying the most likely value categories from textual arguments in English. Our approach includes an ensemble comprising three models: (i) An entailment-based model for determining the human values based on their descriptions, (ii) A Roberta-based classifier that predicts the set of human values from an ar- gument. (iii) A Roberta-based classifier to pre- dict a reduced set of human values from an argu- ment. Compared to standard classification-based approaches like models (ii) and (iii), we observe superior results from the entailment-based model. We experiment with different ways of combining the models1, attaining an overall F1 score of 0.48 on the main test set, which is an improvement over the baseline of 0.42. 2 Background We used the standard training, validation and test splits of the shared-task dataset (Mirzakhmedova et al., 2023) in English, which is based on the Webis-ArgValues-22 dataset (Kiesel et al., 2022). The dataset comprises 9,324 arguments (5,393 train, 1,996 validation, and 1,935 test) from 6 di- verse sources, covering religious texts, political dis- cussions, free-text arguments, newspaper editorials and online democracy …

Content truncated to 500 words