XD at SemEval-2020 Task 12: Ensemble Approach to Offensive Language Identification in Social Media Using Transformer Encoders
24367Authors
nan
Abstract
AbstractThis paper presents six document classification models using the latest transformer encoders and a high-performing ensemble model for a task of offensive language identification in social media. For the individual models, deep transformer layers are applied to perform multi-head attentions. For the ensemble model, the utterance representations taken from those individual models are concatenated and fed into a linear decoder to make the final decisions. Our ensemble model outperforms the individual models and shows up to 8.6% improvement over the individual models on the development set. On the test set, it achieves macro-F1 of 90.9% and becomes one of the high performing systems among 85 participants in the sub-task A of this shared task. Our analysis shows that although the ensemble model significantly improves the accuracy on the development set, the improvement is not as evident on the test set.
Keywords
Full Content
Proceedings of the 14th International Workshop on Semantic Evaluation, pages 239–245 Barcelona, Spain (Online), December 12, 2020. 239 UoB at SemEval-2020 Task 1: Automatic Identification of Novel Word Senses Eleri Sarsfield School of Computer Science University of Birmingham United Kingdom elerisarsfield@gmail.com Harish Tayyar Madabushi School of Computer Science University of Birmingham United Kingdom H.TayyarMadabushi.1@bham.ac.uk Abstract Much as the social landscape in which languages are spoken shifts, language too evolves to suit the needs of its users. Lexical semantic change analysis is a burgeoning field of semantic analysis which aims to trace changes in the meanings of words over time. This paper presents an approach to lexical semantic change detection based on Bayesian word sense induction suitable for novel word sense identification. This approach is used for a submission to SemEval-2020 Task 1, which shows the approach to be capable of the SemEval task. The same approach is also applied to a corpus gleaned from 15 years of Twitter data, the results of which are then used to identify words which may be instances of slang. 1 Introduction Automatic lexical semantic change detection is a field of semantic analysis which aims to discern how the meanings of words change over time. As interest in the field has increased, a variety of different procedures, languages and corpora have been used, which leads to difficulty when attempting to compare different sets of results. SemEval-2020 Task 1, “Unsupervised Lexical Semantic Change Detection” (Schlechtweg et al., 2020), is a task aimed at providing a single unified framework with which to compare approaches using a standardised dataset, in order to address the difficulty in attempting to compare different sets of results which arises due to the variety in procedures, languages and corpora which have previously been used. The task involves determining whether a set of target words have changed meaning in two corpora, each of which corresponds to a different time period. Corpora are provided for German, English, Latin and Swedish. The task consists of two subtasks, one involving a binary classification of target words into words which have or have not changed meaning, the other involving a ranking of words according to degree of change. This paper describes our submission to SemEval-2020 Task 11. To produce this submission, we use an approach based on the Hierarchical Dirichlet Process (HDP) (Teh et al., 2004), an extension of Latent Dirichlet Allocation (LDA) which allows the number of senses to be unbounded. The next section presents works related to methods for identifying novel word senses, which is followed by a description of our system in Section 3 and the results of the application of the system in Section 4. Section 5 provides an overview of our conclusions and possible directions of future work. We also use our method on a corpus constructed from data from Twitter in order to explore the possibility of detecting semantic change over a shorter period of time, with a focus on where that detection can be used for identifying slang. 2 Related Work …
Content truncated to 500 words
Labeled Text
AbstractThis paper presents six document classification models using the latest transformer encoders and a high-performing ensemble model for a task of offensive language identification in social media. For the individual models, deep transformer layers are applied to perform multi-head attentions. For the ensemble model, the utterance representations taken from those individual models are concatenated and fed into a linear decoder to make the final decisions. Our ensemble model outperforms the individual models and shows up to 8.6% improvement over the individual models on the development set. On the test set, it achieves macro-F1 of 90.9% and becomes one of the high performing systems among 85 participants in the sub-task A of this shared task. Our analysis shows that although the ensemble model significantly improves the accuracy on the development set, the improvement is not as evident on the test set.
Publication Details
- Paper ID:
- 24367
- Year:
- 2020
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan