RNRE-NLP at SemEval-2022 Task 4: Patronizing and Condescending Language Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 29 - 35 July 14-15, 2022 ©2022 Association for Computational Linguistics LingJing at SemEval-2022 Task 1: Multi-task Self-supervised Pre-training for Multilingual Reverse Dictionary Bin Li1∗, Yixuan Weng2∗, Fei Xia2,3∗, Bin Sun1, Shutao Li1, Shizhu He2,3 1 College of Electrical and Information Engineering, Hunan University 2 National Laboratory of Pattern Recognition, Institute of Automation, CAS 3 School of Artificial Intelligence, University of Chinese Academy of Sciences {libincn, shutao_li, sunbin611}@hnu.edu.cn, wengsyx@gmail.com, xiafei2020@ia.ac.cn, shizhu.he@nlpr.ia.ac.cn Abstract This paper introduces the result of Team LingJing’s experiments in SemEval-2022 Task 1 Comparing Dictionaries and Word Embed- dings (CODWOE)1. This task aims at compar- ing two types of semantic descriptions, includ- ing the definition modeling and reverse dictio- nary track. Our team focuses on the reverse dictionary track and adopts the multi-task self- supervised pre-training for multilingual reverse dictionaries. Specifically, the randomly initial- ized mDeBERTa-base model is used to per- form multi-task pre-training on the multilin- gual training datasets. The pre-training step is divided into two stages, namely the MLM pre- training stage and the contrastive pre-training stage. As a result, all the experiments are performed on the pre-trained language model during fine-tuning. The experimental results show that the proposed method has achieved good performance in the reverse dictionary track, where we rank the 1-st in the Sgns tar- gets of the EN and RU languages. All the ex- perimental codes are open-sourced at https: //github.com/WENGSYX/Semeval. 1 Introduction The CODWOE shared task invites the participants to compare two types of semantic descriptions: dic- tionary glosses and word embedding representa- tions. The intuitions come from the questions: “Are these two types of representation equivalent? Can we generate one from the other?”. To study this question, the CODWOE proposes two sub-tracks: a definition modeling track (Noraset et al., 2017), where participants have to generate glosses from vectors, and a reverse dictionary track (Hill et al., 2016), where participants have to generate vectors from glosses. These two tracks are fairly challeng- ing (Hill et al., 2016), where more efficient meth- ods are required to be designed for implementation. ∗These authors contribute equally to this work. 1https://codwoe.atilf.fr/ word Embedding sgns: [0.9233651757, -0.526638031, 2.0892603397, … …]256 char: [-0.1214295924, -0.2428643405, 0.2625943422, … …]256 electra: [0.3655579984, -0.1910238415, 0.0170905143,… …]256 " A meal consisting of food normally eaten in the morning , which may typically include eggs , sausages , toast , bacon , etc. " word Gloss Figure 1: An example of the reverse dictionary task. Given the word gloss, it is required to generate the vectors of their corresponding Sgns, Char, and Electra, respectively. These tasks are also useful for explainable AI, since they involve converting human-readable data into machine-readable data and back (Li et al., 2021). In this paper, we focus on the reverse dictionary track. As shown in Figure 1, given the gloss “A meal consisting of food normally eaten in the morn- ing, which may typically include eggs, sausages, toast, bacon, etc.”, the reverse dictionary task re- quires us …

Content truncated to 500 words