Andronicus of Rhodes at SemEval-2023 Task 4: Transformer-Based Human Value Detection Using Four Different Neural Network Architectures

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 542–548 July 13-14, 2023 ©2023 Association for Computational Linguistics Andronicus of Rhodes at SemEval-2023 Task 4: Transformer-Based Human Value Detection using Four Different Neural Network Architectures Georgios Papadopoulos University of Piraeus, Dept. of Digital Systems, Piraeus, 18435, Greece georgepap@unipi.gr Marko Kokol Semantika Research, Semantika d.o.o., Zagrebška 40a, 2000 Maribor, Slovenia marko.kokol@semantika.eu Maria Dagioglou Inst. of Informatics & Telecommunications, National Centre for Scientific Research (N.C.S.R.) “Demokritos” Aghia Paraskevi, Attica, Greece mdagiogl@iit.demokritos.gr Georgios Petasis Inst. of Informatics & Telecommunications, National Centre for Scientific Research (N.C.S.R.) “Demokritos” Aghia Paraskevi, Attica, Greece petasis@iit.demokritos.gr Abstract This paper presents our participation to the “Hu- man Value Detection” shared task (Kiesel et al., 2023), as “Andronicus of Rhodes”. We de- scribe the approaches behind each entry in the official evaluation, along with the motivation behind each approach. Our best-performing approach has been based on BERT large, with 4 classification heads, implementing two dif- ferent classification approaches (with different activation and loss functions), and two different partitioning of the training data, to handle class imbalance. Classification is performed through majority voting. The proposed approach out- performs the BERT baseline, ranking in the upper half of the competition. 1 Introduction The “Human Value Detection” shared task (Kiesel et al., 2023) relates to the identification of values that can be associated with an argument. In the context of the shared task, a single argument is represented by a single premise, a conclusion, and information on what the stance of the premise is towards the conclusion. Each argument can be clas- sified into one or more value categories, among the 20 value categories, based on Schwartz (1994). The shared task covers a single language, English, providing 5393 examples for training, 1896 exam- ples for validation, while participating systems are evaluated on 1576 test examples (Mirzakhmedova et al., 2023). With the ever-increasing ubiquity of artificial in- telligence in real-world usage scenarios, the aware- ness of the so-called alignment problem (Christian, 2020) is increasing, as is the need to develop novel strategies to measure and verify the alignment of machine learning (ML)/artificial intelligence (AI) systems (Brown et al., 2021) with human values. However, high-quality datasets and novel (moral) value recognition methods are needed to further develop this area. Some existing initiatives presenting datasets for human value detection are based: either on specific types of texts, such as the SemEval argument annotation task (Mirzakhme- dova et al., 2023), or on broader over-arching stud- ies of values and their analysis in different spa- tiotemporal contexts, such as the VAST Project approach (Castano et al., 2021). Nevertheless, since human value recognition in texts is a problem that can be challenging even for humans, let alone for machines (Haas, 2020), despite the existing datasets, additional methods need to be developed to detect values in various scenarios. In order to approach the challenge, several differ- ent natural language processing (NLP) and ML/AI approaches have been previously proposed and tested (Kiesel et al., 2022; Yu et al., …

Content truncated to 500 words