Arthur Caplan at SemEval-2023 Task 4: Enhancing Human Value Detection through Fine-tuned Pre-trained Models

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1953–1959 July 13-14, 2023 ©2023 Association for Computational Linguistics Arthur Caplan at SemEval-2023 Task 4: Enhancing Human Value Detection through Fine-tuning Pre-trained Models Xianxian Song, Jinhui Zhao, Ruiqi Cao, Linchi Sui, Tingyue Guan and Binyang Li∗ Lab of Intelligent Social Computing University of International Relations, Beijing, China {kilo0409, jhZhao, rqCao, lcSui,tyGuan, byli}@uir.edu.cn Abstract The computational identification of human val- ues is a novel and challenging research that holds the potential to offer valuable insights into the nature of human behavior and cog- nition. This paper presents the methodology adopted by the Arthur-Caplan research team for the SemEval-2023 Task 4, which entailed the detection of human values behind argu- ments. The proposed system integrates BERT, ERNIE2.0, RoBERTA and XLNet models with fine tuning. Experimental results show that the macro F1 score of our system achieved 0.512, which overperformed baseline methods by 9.2% on the test set. 1 Introduction Human values refer to the cognition, understanding, judgment, or choice made based on people’s certain thinking and senses, which is a kind of thinking or orientation for people to identify things and de- termine right and wrong, to reflect a certain value or function of people and objects. Human values are studied both in the social sciences (Schwartz, 1994) and formal argumentation (Bench-Capon, 2003) for decades to observe different value pri- orities between cultures and disagreement. Some values tend to conflict and others to align, which can cause disagreement on the forward course and opinion. Several approaches have been developed to study human values in the social sciences. However, the task to identify values in arguments seems chal- lenging due to their large number, often implicit use in arguments, and vague definitions. Thus, this task (Kiesel et al., 2023) is an attempt at the auto- matic identification of values in written arguments. It classifies a given text parameter and a person’s value category as to whether the parameter belongs to the categories. This task uses a set of 20 value ∗*Corresponding author categories compiled from the social science litera- ture from their ACL paper (Kiesel et al., 2022). In this paper, we presented various pre-trained models, such as BERT (Devlin et al., 2018), ERNIE2.0 (Sun et al., 2020), RoBERTa (Liu et al., 2019) and XLNet (Yang et al., 2019) to address each component of this task, with a bunch of fine-tuning and ensemble tech- niques. In particular, we introduced prompt (Liu et al., 2023) in our model. The code of the task has been made publicly available from https://github.com/KiloChips/semeval2023-task4. In this task, our team finally achieved the follow- ing results as shown in Table 1. Task F1 score Rank main test 0.512 6/41 test-Nahjalbalagha 0.285 11/20 test-nyt 0.322 3/12 Table 1: Outcome and rank. The rest of the paper is organized as follows: Sec- tion 2 introduces the background of the emergence and development of the task. Section 3 presents our system and models. Section 4 describes the exper- imental setup. …

Content truncated to 500 words