UPB at SemEval-2020 Task 9: Identifying Sentiment in Code-Mixed Social Media Texts Using Transformers and Multi-Task Learning

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 150–157 Barcelona, Spain (Online), December 12, 2020. 150 MineriaUNAM at SemEval-2020 Task 3: Predicting Contextual Word Similarity Using a Centroid based Approach and Word Embeddings Helena G´omez-Adorno1 Gemma Bel-Enguix1 helena.gomez@iimas.unam.mx gbele@iingen.unam.mx Jorge Reyes-Maga˜na1,2 Benjam´ın Moreno3 jorge.reyes@correo.uady.mx bmm@xanum.uam.mx Ram´on Casillas1 Daniel Vargas1 ramon.casillas@comunidad.unam.mx danizzvargas@gmail.com 1Universidad Nacional Aut´onoma de M´exico, Ciudad de M´exico, M´exico 2 Universidad Aut´onoma de Yucat´an, M´erida, M´exico 3 Universidad Aut´onoma Metropolitana, Ciudad de M´exico, M´exico Abstract This paper presents our systems to solve Task 3 of Semeval-2020, which aims to predict the effect that context has on human perception of similarity of words. The task consists of two subtasks in English, Croatian, Finnish and Slovenian: (1) predicting the change of similarity, and (2) predicting the human scores of similarity, both of them for a pair of words within two different contexts. We tackled the problem by developing two systems, the first one uses a centroid approach and word vectors. The second one uses the ELMo language model, which is trained for each pair of words with the given context. Our approach achieved the highest score in subtask 2 for the English language. 1 Introduction The aim of Semeval 2020 Task 3 (Armendariz et al., 2020), Graded Word Similarity in Context (GWSC), focuses on predicting the effect that context has in human perception of similarity of words. The most common works in this area deal with the context where a word is placed to predict discrete changes in meaning: the different senses of a polysemous word (Castillo et al., 2008; Lossio-Ventura et al., 2016; Wang et al., 2018). However, this task highlights the context with more subtle (graded) effects on meaning, even for words not necessarily considered polysemous. That makes word similarity more challenging to be automatically predicted. The main task is divided into two subtasks: • Subtask 1: Predicting the change in the human annotator’s scores of similarity when presented with the same pair of words within two different contexts. This task directly addresses our main question. It evaluates how well systems are able to model the effect that context has in human perception of similarity. • Subtask 2: Predicting the human scores of similarity for a pair of words within two different contexts. This is a more traditional task which evaluates systems’ ability to model both similarity of words and the effect that context has on it. The datasets provided for this task contained contextual similarity ratings in four different languages: Croatian (HR), English (EN), Finnish (FI), and Slovenian (SL). The pairs of words come from the well-known SimLex999 (Hill et al., 2015) dataset. The rest of the paper is organized as follows: in Section 2 our methodology is presented, along with some preliminary results using the corpus available in the practice phase. This will guide us to improve each approach. The final results of all systems with the evaluation corpus is reported in Section 3. The paper ends with some conclusions in Section 4. This work …

Content truncated to 500 words