LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation Using Pretraining Language Model

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 562–568 Barcelona, Spain (Online), December 12, 2020. 562 LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model Shilei Liu1, Yu Guo1, Bochao Li1 and Feiliang Ren12∗ 1School of Computer Science and Engineering, Northeastern University 2Key Laboratory of Data Analytics and Optimization for Smart Industry (Northeastern University), Ministry of Education {1901750,1871502,1901725}@stu.neu.edu.cn renfeiliang@ise.neu.edu.cn Abstract This paper describes our submission1 to subtask a and b of SemEval-2020 Task 4. For subtask a, we use a ALBERT based model with improved input form to pick out the common sense statement from two statement candidates. For subtask b, we use a multiple choice model enhanced by hint sentence mechanism to select the reason from given options about why a statement is against common sense. Besides, we propose a novel transfer learning strategy between subtasks which help improve the performance. The accuracy scores of our system are 95.6 / 94.9 on official test set2 and rank 7th / 2nd on Post-Evaluation leaderboard. 1 Introduction Common sense verification and explanation is an important and challenging task in artificial intelligence and natural language processing. This is a simple task for human beings, because human beings can make full use of external knowledge accumulated in their daily lives. However, common sense verification and reasoning is difficult for machines. According to Wang (2019), even some state-of-the-art language models such as ELMO(Peters et al., 2018) and BERT(Devlin et al., 2019) have very poor performance. So it is crucial to integrate the ability of commonsense-aware to natural language understanding model(Davis, 2017). SemEval-2020 task4(Wang et al., 2020) aims to improve the ability of common sense judgment for model, and we participated in two subtasks of this task. The dataset of SemEval-2020 task4 named ComVE. Each instance in ComVE is composed of 5 sentences ⟨s1, s2, o1, o2, o3⟩. s1 and s2 will be used for subtask a, and s1 or s2 with ⟨o1, o2, o3⟩ will be used for subtask b. Subtask a(also known as Sen-Making task) aims to test a model’s ability of commonsense validation. Specifically, given two statements ⟨s1, s2⟩ whose lexical and syntactic are similar, the object of Sen- Making model is to determine which statement is common sense(compared to another one). For example, s1 is put the elephant in the refrigerator and s2 is put the turkey in the refrigerator, a good model needs to judge that the latter is more common sense. Subtask b(also known as Explanation task) is a multiple choice task that aims to find the key reason why a given statement does not make sense. For example, given a sentence that violates common sense with three options ⟨s, o1, o2, o3⟩, where s is he put an elephant into the fridge, o1 is an elephant is much bigger than a fridge, o2 is elephants are usually gray while fridges are usually white, and o3 is an elephant cannot eat a fridge, the model needs to judge that o1 is the correct …

Content truncated to 500 words