TueFact at SemEval 2019 Task 8: Fact checking in community question answering forums: context matters

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1176–1179 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1176 TueFact at SemEval 2019 Task 8: Fact checking in community question answering forums: context matters R´eka Juh´asz University of T¨ubingen Wilhelmstr. 19-23 72074 T¨ubingen, Germany reka.juhasz@student .uni-tuebingen.de Franziska Barbara Linnenschmidt University of T¨ubingen Wilhelmstr. 19-23 72074 T¨ubingen, Germany franziska-barbara. linnenschmidt@student .uni-tuebingen.de Teslin Roys University of T¨ubingen Wilhelmstr. 19-23 72074 T¨ubingen, Germany teslin.roys@student .uni-tuebingen.de Abstract The SemEval 2019 Task 8 on Fact-Checking in community question answering forums aimed to classify questions into categories and ver- ify the correctness of answers given on the QatarLiving public forum. The task was di- vided into two subtasks: the first classifying the question, the second the answers. The Tue- Fact system described in this paper used differ- ent approaches for the two subtasks. Subtask A makes use of word vectors based on a bag- of-word-ngram model using up to trigrams. Predictions are done using multi-class logistic regression. The official SemEval result lists an accuracy of 0.60. Subtask B uses vector- ized character n-grams up to trigrams instead. Predictions are done using a LSTM model and achieved an accuracy of 0.53 on the final Se- mEval Task 8 evaluation set. In a comparison of contextual inputs to subtask B, it was de- termined that more contextual data improved results, but only up to a point. 1 Introduction The SemEval 2019 Task 8 on Fact-Checking gave us the opportunity to develop a system that evalu- ates the factual content of questions and answers in the field of community question answering fo- rums. This popular niche on the internet pro- vides helpful information for specific interests, such as information on life in Qatar or elsewhere in the world, coding help on Stack Overflow, or answers to a wide range of questions on Quora, Reddit/r/Ask or Yahoo! Answers. Often other re- sources are not at hand or misleading, and it proves difficult to find what one is looking for amid non- relevant questions, let alone be sure the answers found are correct. A system that can, with some degree of accuracy, pick out the factual questions and then predict the correctness of the given an- swers, is a means to ensure quality in community question answering forums. There may also be many further applications in information retrieval – ordering search results based on estimated fac- tuality in a web query or even identifying truthful answers in automatic question answering systems. The task was divided into two subtasks. While Subtask A required a classification of the ques- tions into three distinct categories “Factual”, “So- cializing”, and “Opinion”, Subtask B took the sub- set of the “Factual” questions to classify them ac- cording to either “True”, “False” or “NonFactual” in terms of the actual answer. Similar tasks were already part of SemEval 2015 (Nakov et al., 2015) and SemEval 2016 (Nakov et al., 2016). The Tuefact system follows this division, even going as far as using different …

Content truncated to 500 words