YNU-oxz at SemEval-2020 Task 4: Commonsense Validation Using BERT with Bidirectional GRU

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 626–632 Barcelona, Spain (Online), December 12, 2020. 626 YNU-oxz at SemEval-2020 Task 4: Commonsense Validation using BERT with Bidirectional GRU Xiaozhi Ou, Hongling Li ∗ School of Information Science and Engineering Yunnan University, Yunnan, P.R. China Abstract This paper describes the system and results of our team participated in SemEval-2020 Task4: Commonsense Validation and Explanation (ComVE), which aim to distinguish meaningful nat- ural language statements from unreasonable natural language statements. This task contains three subtasks: Subtask A–Validation, Subtask B–Explanation (Multi-Choice), and Subtask C– Explanation (Generation). In these three subtasks, we only participated in Subtask A, which aims to distinguish whether a given two natural language statements with similar wording are meaningful. To solve this problem, we proposed a method using a combination of BERT with the Bidirectional Gated Recurrent Unit (Bi-GRU). Our model achieved an accuracy of 0.836 in Subtask A (ranked 27/45). 1 Introduction In recent years, the introduction of commonsense into natural language understanding systems has re- ceived more and more research attention. Commonsense Validation and Explanation describes whether the natural language statement is consistent with the facts and explains the reasons for its against com- mon sense. Specifically, the purpose of the commonsense verification is to determine that the natural language statement is reasonable and meaningful. This task involves commonsense, understanding, and reasoning, so it is very important for a system to be able to accurately distinguish between meaningful natural language statements and unreasonable natural language statements. In SemEval-2020 Task 4: Verification and Interpretation of Common Sense (Wang et al., 2020). The organizer has designed three subtasks. Subtask A selects from two natural language statements with similar grammatical structures (one of which is meaningful and one of which is not). It requires the model to determine which of the two statements violates commonsense; Subtask B is to find the key reason why a given statement violates commonsense from three options, this task is a multiple-choice; Subtask C requires the system to gen- erate the reason why this statement violates commonsense. Among these three subtasks, Subtask A and Subtask B are evaluated by accuracy, and Subtask C is evaluated by BLEU. For Subtask A, we exper- imented with different neural network-based models, such as Bidirectional Long Short-Term Memory (Bi-LSTM) (Hochreiter and Schmidhuber, 1997), Bidirectional Gated Recurrent Unit (Bi-GRU) (Bah- danau et al., 2014), and bidirectional encoder representation from Transformers (BERT) (Devlin et al., 2018). We found that BERT performs best on the validation set. Therefore, we chose to use BERT to combine different models to optimize the system. The rest of the paper is structured as follows. Section 2 introduce the related work of ComVE; section 3 introduce the data description and methods; in section 4, we analyze the experimental results; in section 5, we summarize and look forward to future work. ∗Corresponding author: honglingli66@126.com This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http:// creativecommons.org/licenses/by/4.0/ 627 2 Related Work Commonsense verification aims …

Content truncated to 500 words