TeamJUST at SemEval-2020 Task 4: Commonsense Validation and Explanation Using Ensembling Techniques

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 594–600 Barcelona, Spain (Online), December 12, 2020. 594 TeamJUST at SemEval-2020 Task 4: Commonsense Validation and Explanation Using Ensembling Techniques Roweida Mohammed and Malak Abdullah Jordan University of Science and Technology Irbid, Jordan roweida.221@gmail.com mabdullah@just.edu.jo Abstract Common sense for natural language processing methods has been attracting a wide research interest, recently. Estimating automatically whether a sentence makes sense or not is considered an essential question. Task 4 in the International Workshop SemEval 2020 has provided three subtasks (A, B, and C) that challenges the participants to build systems for distinguishing the common sense statements from those that do not make sense. This paper describes TeamJUST’s approach for participating in subtask A to differentiate between two sentences in English and classify them into two classes: common sense and uncommon sense statements. Our approach depends on ensembling four different state-of-the-art pre-trained models (BERT, ALBERT, Roberta, and XLNet). Our baseline model which we used only the pre-trained model of BERT has scored 89.1, while the TeamJUST model outperformed the baseline model with an accuracy score of 96.2. We have improved the results in the post-evaluation period to achieve our best result, which would rank the 4th in the competition if we had the chance to use our latest experiment. 1 Introduction Natural Language Processing and Understanding (NLP/NLU) have great care of research, recently. Several language models had been trained on huge data of corpora (Peters et al., 2018; Devlin et al., 2018), and some benchmarks showed how methods display enhanced performance (Devlin et al., 2018). However, most end-to-end trained methods on common sense, compared to people, are disappointing. For instance, people can understand and recognize immediately that we can place an apple into a fridge, but cannot and never place a TV into a fridge. Thus, for trained systems, it is much harder to recognize the difference. Accordingly, it is essential to have the ability to evaluate how good a system is to recognize the sense (Davis, 2017). Recent datasets have experimented common sense over tasks, for example co-reference resolution (Ortiz, 2015) or subsequent event prediction (Zellers et al., 2018). They stated that a method is prepared with common sense through testing if it can offer a correct answer where no added knowledge to the input. SemEval is the International Workshop on Semantic Evaluation, which is developed from SensEval to evaluate semantic study methods. The 14th version of this workshop, SemEval-2020, has provided 12 tasks. We have participated in Task 4 - Commonsense Validation and Explanation (Wang et al., 2020), which offers three subtasks with datasets in English. The main goal of this task is to recognize which sentences are common sense. Further information about Task 4 and the datasets is given in Section 3. This paper describes our model for participating in SemEval-2020 (Task 4 - Sub Task A). Our team has built an ensembling system to select from two English statements that both have alike words to identify which one is …

Content truncated to 500 words