BLCU_NLP at SemEval-2019 Task 8: A Contextual Knowledge-enhanced GPT Model for Fact Checking

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1132–1137 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1132 BLCU NLP at SemEval-2019 Task 8: A Contextual Knowledge-enhanced GPT Model for Fact Checking Wanying Xie, Mengxi Que, Ruoyao Yang, Chunhua Liu, Dong Yu(�) Beijing Language and Culture University, Beijing, China {xiewanying07, quemengxi, yangruoyao97, chunhualiu596}@gmail.com yudong@blcu.edu.cn Abstract Since the resources of Community Question Answering are abundant and information shar- ing becomes universal, it will be increasingly difficult to find factual information for ques- tioners in massive messages. SemEval 2019 task 8 is focusing on these issues. We par- ticipate in the task and use Generative Pre- trained Transformer (OpenAI GPT) as our sys- tem. Our innovations are data extension, fea- ture extraction, and input transformation. For contextual knowledge enhancement, we ex- tend the training set of subtask A, use several features to improve the results of our system and adapt the input formats to be more suitable for this task. We demonstrate the effectiveness of our approaches, which achieves 81.95% of subtask A and 61.08% of subtask B in accu- racy on the SemEval 2019 task 8. 1 Introduction With the development of Community Question Answering (cQA) forums, massive information is being shared. However, not all information is factual, which makes finding an appropriate an- swer to satisfy the information needs of question- ers more difficult. Previous work which concen- trated on these problems (Nakov et al., 2017) re- ranked the questions based on their relevance with the original question. ˇSaina et al. (2017) treated the similarity ranking task as a binary classifica- tion problem. We study these issues in SemEval- 2019 Task 8 (Mihaylova et al., 2019) by using the contextual Knowledge-enhanced GPT (Rad- ford et al., 2018), which use Transformer (Vaswani et al., 2017) as model architecture. The contextual knowledge enhancement includes data extension, feature extraction, and input transformation. The task includes two subtasks and they are both three classification problems. In subtask A, we need to find out whether a question seeks a factual answer, an opinion or just want to socialize with others. We classify the answers for questions that look for factual information in subtask A into three classes in subtask B: true, false or nonfac- tual. In this paper, we study both subtasks and use a similar system to solve them. Several challenges exist when doing this task. The size of datasets for both subtasks is small. The data contains a number of complex long text, which makes extracting key information more dif- ficult. The input format of GPT changes with dif- ferent objectives of tasks, so it requires some mod- ifications to fit specific tasks. We apply three points to solve these problems. We extend the training set of subtask A from two other datasets: DailyDialog (Li et al., 2017) and SQuAD2.0 (Rajpurkar et al., 2018). We use two methods to guarantee the quality of expanded datasets. Firstly we use the Levenshtein Distance to screen similar data, and then …

Content truncated to 500 words