Team_KGP at SemEval-2021 Task 7: A Deep Neural System to Detect Humor and Offense with Their Ratings in the Text Data

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1169–1174 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1169 Team KGP at SemEval-2021 Task 7: A Deep Neural System to Detect Humor and Offense with Their Ratings in the Text Data Anik Mondal Indian Institute of Technology Kharagpur pjanik2000@iitkgp.ac.in Raksha Sharma Indian Institute of Technology Roorkee raksha.sharma@cs.iitr.ac.in Abstract This paper describes the system submitted to SemEval-2021 Task-7 for all four subtasks. Two subtasks focus on detecting humor and of- fense from the text (binary classification). On the other hand, the other two subtasks predict humor and offense ratings of the text (linear regression). In this paper, we present two dif- ferent types of fine-tuning methods by using linear layers and bi-LSTM layers on top of the pre-trained BERT model. Results show that our system is able to outperform baseline models by a significant margin. We report F1 scores of 0.90 for the first subtask and 0.53 for the third subtask, while we report an RMSE of 0.57 and 0.58 for the second and fourth sub- tasks, respectively. 1 Introduction Automatic humor and offense detection have con- siderable importance in the modern age, especially in chatbots and virtual assistants (Augello et al., 2008). Detection of humor and offense in the text is a highly subjective phenomenon that varies with age, gender, race, socio-economic status, etc. Therefore, it is one of the most challenging re- search fields in Natural Language Processing (Tay- lor, 2009). Task-7 of SemEval 2021 (Meaney et al., 2021) is concerned with humor detection (binary classi- fication) primarily. The first subtask is to check if a text is humorous, another subtask follows up: how humorous is the text (rated from 0-5). We also have to predict whether the text is generally offen- sive (binary classification task). The last subtask predicts how generally offensive a text is (rated from 0-5), regardless of whether it is classed as humorous or offensive overall. The dataset consists entirely of English texts. The traditional methods deployed earlier for hu- mor detection include Support Vector Machine (SVM) with RBF kernel, Random Forest Classi- fier, and SGD with Logical Classifier, all of which provide modest results (de Oliveira and Rodrigo, 2015). Recently more state-of-the-art transform- ers have provided better results (Weller and Seppi, 2019). The system presented in this paper is fine- tuning one of the best and most popular state-of-the- art models, Bidirectional Encoder Representations from Transformers (BERT). We have used pre- trained BERT embeddings to represent the words and used the features of the last layer of the 12- layers BERT Model to detect and rate both humor and offense. BERT can model complex interactions between different levels of hierarchical information (Tenney et al., 2019). This task is perhaps the first task that focuses on the humor and offense ratings and combines humor and offense detection. Our model obtained an F1 score of 0.92 and 0.56 for the first and the third subtasks, respectively, on the test data. We …

Content truncated to 500 words