Voice@SRIB at SemEval-2020 Tasks 9 and 12: Stacked Ensemblingmethod for Sentiment and Offensiveness detection in Social Media

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1331–1341 Barcelona, Spain (Online), December 12, 2020. 1331 Voice@SRIB at SemEval-2020 Tasks 9 and 12: Stacked Ensembling method for Sentiment and Offensiveness detection in Social Media Abhishek Singh Samsung R&D Bangalore abhi3.singh@samsung.com Surya Pratap Singh Parmar Samsung R&D Bangalore s.singhparm@samsung.com Abstract In social-media platforms such as Twitter, Facebook, and Reddit, people prefer to use code-mixed language such as Spanish-English, Hindi-English to express their opinions. In this paper, we describe different models we used, using the external dataset to train embeddings, ensembling methods for Sentimix, and OffensEval tasks. The use of pre-trained embeddings usually helps in multiple tasks such as sentence classification, and machine translation. In this experiment, we have used our trained code-mixed embeddings and twitter pre-trained embeddings to SemEval tasks. We evaluate our models on macro F1-score, precision, accuracy, and recall on the datasets. We intend to show that hyper-parameter tuning and data pre-processing steps help a lot in improving the scores. In our experiments, we are able to achieve 0.886 F1-Macro on OffenEval Greek language subtask post-evaluation, whereas the highest is 0.852 during the Evaluation Period. We stood third in Spanglish competition with our best F1-score of 0.756. Codalab username is asking28. 1 Introduction SemEval Task-9 Sentimix (Patwa et al., 2020) is divided into two tasks, one for Hinglish (Hindi-English) and the other for Spanglish (Spanish-English) code-mixed subtasks. In the Spanglish task, the dataset contains tweets in Spanglish (Spanish-English) code-mixed language, and it is labeled into three categories positive, negative, and neutral sentiments. The task is to classify codemixed tweets into these three sentiments. SemEval Task-12 OffensEval (Zampieri et al., 2020) is divided into different subtasks, English (Rosenthal et al., 2020), Danish (Sigurbergsson and Derczynski, 2020), Arabic (Mubarak et al., 2020), Turkish (C¸ ¨oltekin, 2020), and Greek (Pitenis et al., 2020) languages. English task is divided into three subtasks- A, B, and C. Subtask-A of OffensEval is offensive language identification, subtask- B is categorization of offensive types into targeted and untargeted, and subtask-C is offensive target identification as individual, group, or other. In the last decade, there has been proliferation in the use of social media web sites. It has led to pervasive use of hate inducing speech and offensive language to express opinions. The use of profane language has been growing in face-to-face interactions as well as online communications in recent years. The anonymity provided by these websites and lack of stringent action has led to adoption aggressive behavior by people.Youth who experienced cyberbullying, as either an offender or a victim, had more suicidal thoughts and were more likely to attempt suicide than those who had not experienced such forms of peer aggression (Hinduja and Patchin, 2010). Hence it’s necessary to auto-remove offensive and profane language in an online environment. Since the inflow of such type of content is huge, manual filtering is time-consuming and requires much manual labor; hence it becomes almost impractical to do manual filtering. Due to this reason, researchers have proposed methods to …

Content truncated to 500 words