IIE-NLP-NUT at SemEval-2020 Task 4: Guiding PLM with Prompt Template Reconstruction Strategy for ComVE

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 346–353 Barcelona, Spain (Online), December 12, 2020. 346 IIE-NLP-NUT at SemEval-2020 Task 4: Guiding PLM with Prompt Template Reconstruction Strategy for ComVE Luxi Xing, Yuqiang Xie, Yue Hu∗, Wei Peng Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China {xingluxi,xieyuqiang,huyue,pengwei}@iie.ac.cn Abstract This paper introduces our systems for the first two subtasks of SemEval Task4: Commonsense Validation and Explanation. To clarify the intention for judgment and inject contrastive information for selection, we propose the input reconstruction strategy with prompt templates. Specifically, we formalize the subtasks into the multiple-choice question answering format and construct the input with the prompt templates, then, the final prediction of question answering is considered as the result of subtasks. Experimental results show that our approaches achieve significant performance compared with the baseline systems. Our approaches secure the third rank on both official test sets of the first two subtasks with an accuracy of 96.4 and an accuracy of 94.3 respectively. 1 Introduction Natural Language Understanding (NLU) requires the systems can not only figure out the semantic of the text but also comprehend text with the constraint of commonsense knowledge about the world. The ability to identify the natural language statement against common sense and produce the explanation of its fault is the foundation towards realizing natural language understanding (Wang et al., 2019c). The SemEval-2020 Task 4 provides a well-formed evaluation mission that aims to evaluate the capacity of the system on commonsense validation and explanation (Wang et al., 2020). The Commonsense Validation and Explanation (ComVE) task is divided into three subtasks including validation, explanation selection, and explanation generation. We mainly focus on and participate in the first two subtasks. The goal of the first validation subtask (subtaskA) is to inspect the ability of a system about distinguishing natural language statements that are against commonsense (i.e. false statements for short). The goal of the second explanation selection subtask (subtaskB) is to test if the system can correctly understand the reason for making against common sense. In subtaskA, the challenge with distinguishing the false statements lies in that this kind of statement usually conforms to the linguistic structure in syntactic level but its meaning does not fit the general commonsense in semantic level. In subtaskB, the difficulty of selecting an appropriate explanation for false statements is that, albeit the candidate explanations are relevant to the content of the false statements, they may not contain the main reason to account for the false statements and will distract the system. To address the above challenges, we first formalize both subtasks as a type of multiple-choice Question Answering (QA) task. Recently, the the large Pre-trained Language Models (PLMs), such as GPT (Radford et al., 2018), BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), demonstrate its excellent ability in various natural language understanding tasks (Levesque et al., 2012; Zellers et al., 2018; Wang et al., 2019b; Wang et …

Content truncated to 500 words