Tenzin-Gyatso at SemEval-2023 Task 4: Identifying Human Values behind Arguments Using DeBERTa

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2062–2066 July 13-14, 2023 ©2023 Association for Computational Linguistics Tenzin-Gyatso at SemEval-2023 Task 4: Identifying Human Values behind Arguments using DeBERTa Pavan Kandru, Bhavyajeet Singh, Ankita Maity, Aditya Hari and Vasudeva Varma IIIT Hyderabad { siri.venkata, bhavyajeet.singh,ankita.maity,aditya.hari}@research.iiit.ac.in vv@iiit.ac.in Abstract Identifying human values behind arguments is a complex task which requires understanding of premise, stance and conclusion together. We propose a method that uses a pre-trained lan- guage model, DeBERTa, to tokenize and con- catenate the text before feeding it into a fully connected neural network. We also show that leveraging the hierarchy in values improves the performance by .14 F1 score compared to only using level 2 values. Our code is made publicly available here.1 1 Introduction Humans often come to different conclusions given the same premise. This variation can be attributed to their values. Identifying the values behind the ar- guments is helpful in understanding the argument it- self. Downstream tasks like supporting or opposing argument generation can benefit from value iden- tification. In this task(Kiesel et al., 2023), we aim to identify 20 value categories in a given premise, stance and conclusion pair. Data used is collected from four geographical cultures, manually anno- tated by (Mirzakhmedova et al., 2023). This paper proposes a method for multi-label classification of the premise, stance, and conclu- sion pairs using Encoder only LMs as a background model. We use DeBERTa(He et al., 2021), a pre- trained language model, that has shown remarkable success in various NLP tasks, including classifica- tion. The proposed method tokenizes the premise, stance and conclusion text using the pretrained to- kenizer, and then concatenates them and feeds it into the LM, generates a representation of the com- bined text, and maps it to a set of values using a fully connected Neural Network. The model is trained on a Multi-margin loss function and evalu- ated on metrics such as accuracy, precision, recall, and F1 score. This approach can potentially result 1https://github.com/pavankandru/values Figure 1: Values in the data organized higher level to lower level. in a highly accurate and effective NLP model for identification of values in arguments. Our major finding from the task is that the hierar- chy in the values helps with identifying them. We found that adding 5 high level values Self-direction, Power, Security, Conformity, Benevolence, Uni- versalism to the existing 20 values improved per- formance substantially compared to just using the 20. These 5 values are extracted from labels of 20 values. For example, the value Self-direction: action belongs to the coarse Self-direction class. Our model outperformed the 1-Baseline and Ran- dom Baseline in (Fröbe et al., 2023) and performed similarly to the BERT Baseline. 2 Background The task involves predicting what values would have lead humans to come to the given stance and conclusion when provided with a premise. There are a total of 20 such values that needs to be identi- 2062 Transformer Layer 1 CLS premise SEP stance SEP conclusion …

Content truncated to 500 words