UoR at SemEval-2020 Task 8: Gaussian Mixture Modelling (GMM) Based Sampling Approach for Multi-modal Memotion Analysis
24504Authors
nan
Abstract
AbstractMemes are widely used on social media. They usually contain multi-modal information such as images and texts, serving as valuable data sources to analyse opinions and sentiment orientations of online communities. The provided memes data often face an imbalanced data problem, that is, some classes or labelled sentiment categories significantly outnumber other classes. This often results in difficulty in applying machine learning techniques where balanced labelled input data are required. In this paper, a Gaussian Mixture Model sampling method is proposed to tackle the problem of class imbalance for the memes sentiment classification task. To utilise both text and image data, a multi-modal CNN-LSTM model is proposed to jointly learn latent features for positive, negative and neutral category predictions. The experiments show that the re-sampling model can slightly improve the accuracy on the trial data of sub-task A of Task 8. The multi-modal CNN-LSTM model can achieve macro F1 score 0.329 on the test set.
Keywords
Full Content
Proceedings of the 14th International Workshop on Semantic Evaluation, pages 135–141 Barcelona, Spain (Online), December 12, 2020. 135 BMEAUT at SemEval-2020 Task 2: Lexical entailment with semantic graphs ´Ad´am Kov´acs∗, Kinga G´emes Dept. of Automation and Applied Informatics Budapest University of Technology and Economics kovacs.adam@aut.bme.hu gemes.kingaandrea@aut.bme.hu Andras Kornai SZTAKI Institute of Computer Science kornai@sztaki.hu G´abor Recski TU Wien gabor.recski@tuwien.ac.at Abstract In this paper we present a novel rule-based, language independent method for determining lexical entailment relations using semantic representations built from Wiktionary definitions. Combined with a simple WordNet-based method our system achieves top scores on the English and Italian datasets of the Semeval-2020 task “Predicting Multilingual and Cross-lingual (graded) Lexical Entailment” (Glavaˇs et al., 2020). A detailed error analysis of our output uncovers future directions for improving both the semantic parsing method and the inference process on semantic graphs. reported in this document. 1 Introduction We present a rule-based, multilingual system for detecting monolingual binary entailment between pairs of words using Wiktionary definitions, dependency parsing, and semantic graphs. We define entailment over pairs of semantic graphs built from dictionary definitions and achieve near-perfect precision on the Semeval-2020 task ”Predicting Multilingual and Cross-lingual (graded) Lexical Entailment” (Glavaˇs et al., 2020), where we participate in the ANY track of the monolingual task that allows for the use of external lexico-semantic resources. Our system improves the performance of strong WordNet-based baselines on three languages, achieving top results on English and Italian and second-best on German. Our pipeline can be easily extended to support any language given a monolingual dictionary and a Universal Dependency (UD) parser. A detailed error analysis shows multiple directions for further improvement, most notably the refinement of the mechanism responsible for recursively extending semantic graphs based on the definition of its nodes. Section 2 briefly describes the lexical entailment task, the 4lang semantic representation (Kornai et al., 2015), and the dict to 4lang tool (Recski, 2016) for generating graphs from dictionary definitions (Recski, 2016; Recski, 2018). Section 3 outlines the architecture of our current system and presents our method for detecting entailment over pairs of 4lang graphs. Our results on the shared task and a detailed error analysis is presented in Section 4. Our system is available for download under an MIT license from GitHub under https://github.com/adaamko/wikt2def/tree/semeval. 2 Background 2.1 Lexical Entailment A common definition of lexical entailment, used also for the current shared task, is that of recognizing IS A relationships between pairs of words (e.g. lettuce entails food). Datasets used in this shared task are derived from the HyperLex dataset (Vuli´c et al., 2017), methods for measuring multi-lingual and cross-lingual lexical entailment using specialized word embeddings are presented in (Vuli´c et al., 2019) and outperform previous baselines in (Upadhyay et al., 2018). Another common lexical inference task ∗corresponding author This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. 136 is defined between pairs of predicates in context, where context can be defined as pairs of arguments (Zeichner et al., 2012), pairs …
Content truncated to 500 words
Labeled Text
AbstractMemes are widely used on social media. They usually contain multi-modal information such as images and texts, serving as valuable data sources to analyse opinions and sentiment orientations of online communities. The provided memes data often face an imbalanced data problem, that is, some classes or labelled sentiment categories significantly outnumber other classes. This often results in difficulty in applying machine learning techniques where balanced labelled input data are required. In this paper, a Gaussian Mixture Model sampling method is proposed to tackle the problem of class imbalance for the memes sentiment classification task. To utilise both text and image data, a multi-modal CNN-LSTM model is proposed to jointly learn latent features for positive, negative and neutral category predictions. The experiments show that the re-sampling model can slightly improve the accuracy on the trial data of sub-task A of Task 8. The multi-modal CNN-LSTM model can achieve macro F1 score 0.329 on the test set.
Publication Details
- Paper ID:
- 24504
- Year:
- 2020
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan