Mao-Zedong at SemEval-2023 Task 4: Label Represention Multi-Head Attention Model with Contrastive Learning-Enhanced Nearest Neighbor Mechanism for Multi-Label Text Classification

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 426–432 July 13-14, 2023 ©2023 Association for Computational Linguistics Mao-Zedong At SemEval-2023 Task 4: Label Represention Multi-Head Attention Model With Contrastive Learning-Enhanced Nearest Neighbor Mechanism For Multi-Label Text Classification Che Zhang+* and Ping’an Liu+ and Zhenyang Xiao* and Haojun Fei+ [+]Qifu Technology,China [*]Peking University mmt,kjn@stu.pku.edu.cn liupingan-jk,zhangchulan-jk@360shuke.com Abstract The study of human values is essential in both practical and theoretical domains. With the development of computational linguistics, the creation of large-scale datasets has made it possible to automatically recognize human values accurately. SemEval 2023 Task 4(Kiesel et al., 2023) provides a set of arguments and 20 types of human values that are implicitly expressed in each argument. In this paper, we present our team’s solution. We use the Roberta(Liu et al.) model to obtain the word vector encoding of the document and propose a multi-head attention mechanism to establish connections between specific labels and semantic components. Furthermore, we use a contrastive learning-enhanced K-nearest neighbor mechanism(Su et al.) to leverage existing instance information for prediction. Our approach achieved an F1 score of 0.533 on the test set and ranked fourth on the leader- board. we make our code publicly available at https://github.com/peterlau0626/semeval2023- task4-HumanValue. 1 Introduction The identification and analysis of human values in texts has been an important area of research. With the development of computational linguistics, this research has gained widespread attention because of its potential impact on areas such as sentiment analysis, social science. One of the challenges in this area is to accurately categorize all the human value. Several notable research achievements have been made in the cate- gorization of human values. One of the approach is that classifies human values into 54 categories across four different levels(Kiesel et al., 2022). Se- mEval2023 task4 uses the classification method in this paper, where an argument is given to iden- tify whether a value is included in the instrument, and the F1 scores of the results at the level2 level are used for the total ranking. There are 20 cate- gories of human values in level 2, and a argument could belong to multiple value categories or not to any one value category. this is a typical multi- label text classification (MLTC) problem which has been applied in many scenarios such as news emotion analysis(Bhowmick et al.) and web page tagging(Jain et al.). In this paper, we propose a model that combines the label-specific attention network with the con- trastive learning-enhanced nearest neighbor mech- anism(Su et al.). The multi-headed attention mech- anism allows our model to overcome the shortcom- ings of traditional attention mechanism models and to be able to focus on different parts of a document, resulting in more accurate labeled attention results. And the nearest neighbor mechanism enables our model to not waste the rich knowledge that can be directly obtained from the existing training in- stances and helps enhance the interpretability and robustness of the model. 2 Background 2.1 Datasets The dataset comprises of arguments from …

Content truncated to 500 words