CodeForTheChange at SemEval-2019 Task 8: Skip-Thoughts for Fact Checking in Community Question Answering

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1138–1143 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1138 CodeForTheChange at SemEval-2019 Task 8: Skip-Thoughts for Fact Checking in Community Question Answering Adithya Avvaru1,3 and Anupam Pandey2,3 1Teradata India Pvt. Ltd, India 2Qubole, India 3 International Institute of Information Technology, Hyderabad, India {adithya.avvaru,anupam.pandey}@students.iiit.ac.in Abstract Community Question Answering (cQA) is one of the popular Natural Language Processing (NLP) problems being targeted by researchers across the globe. Couple of the unanswered questions in the domain of cQA are ‘can we label the questions/answers as factual or not?’ and ‘Is the given answer by the user to a par- ticular factual question is correct and if it is correct, can we measure the correctness and factuality of the given answer?’. We have participated in SemEval-2019 Task 8 which deals with these questions. In this paper, we present the features used, approaches followed for feature engineering, models experimented with and finally the results. Our primary sub- mission with accuracy (official metric for Se- mEval Task 8) of 0.65 in Subtask B (An- swer Classification) and 0.63 in Subtask A (Question Classification) stood at 6th and 16th places respectively. 1 Introduction Community Question Answering (cQA) forums such as Quora, StackOverflow, Yahoo! Answers, Qatar Living etc., now-a-days are fast and effec- tive means of getting answers for any question. But the answers may or may not be correct and factual always. The focus of cQA research, for the last few couple of years, is revolving around deter- mining the model which predicts the best answer for the question, given a question and a number of answers (might be hundreds or even thousands in number). cQA is one of the popular problems being constantly in focus of SemEval organizers since 2015. The subtasks that were targeted earlier in- clude (i) classifying the answer to a particular question as good or potentially good or bad in 20151, (ii) three reranking subtasks i.e., Question- Comment Similarity, Question-Question Similar- ity and Question-External Comment Similarity in 1http://alt.qcri.org/semeval2015/task3/ 20162 and (iii) Question Similarity (QS) to detect duplicate questions and Relevance Classification (RC) in 20173. Contrary to earlier tasks of Se- mEval focusing mainly on classification and sim- ilarity of questions and/or answers and/or com- ments, SemEval-2019 targets the factuality of the questions (whether the question is factual or not) and the factuality of the answers (whether the an- swers provided to the factual questions are factual or not). The tasks become more challenging as data have noisy (like !!!), and unstructured (like Oh..) words. SemEval-2019 Task 8 features the following two subtasks: Subtask A (Question Classification) - determine whether a question asks for a factual informa- tion, an opinion/advice or is just socializing. Example from the “Qatar Living” forum given in competition page4 for this subtask is as follows: Q: I have heard its not possible to extend visit visa more than 6 months? Can U please an- swer me.. Thankzzz... answer 1: Maximum period is 9 …

Content truncated to 500 words