Spider-Jerusalem at SemEval-2019 Task 4: Hyperpartisan News Detection

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 985–989 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 985 Spider-Jerusalem at SemEval-2019 Task 4: Hyperpartisan News Detection Amal Alabdulkarim Department of Computer Science Columbia University amal.a@columbia.edu Tariq Alhindi Department of Computer Science Columbia University tariq@cs.columbia.edu Abstract This paper describes our system for detecting hyperpartisan news articles, which was submitted for the shared task in SemEval 2019 on Hyperpartisan News Detection. We developed a Support Vector Machine (SVM) model that uses TF-IDF of tokens, Language Inquiry and Word Count (LIWC) features, and structural features such as number of paragraphs and hyperlink count in an article. The model was trained on 645 articles from two classes: mainstream and hyperpartisan. Our system was ranked seventeenth out of forty two participating teams in the binary classification task with an accuracy score of 0.742 on the blind test set (the accuracy of the top ranked system was 0.822). We provide a detailed description of our preprocessing steps, discussion of our experiments using different combinations of features, and analysis of our results and prediction errors. 1 Introduction Fake news on various online media outlets misinform the public and threaten the integrity of journalism. This has serious effects on shaping public opinions on controversial topics such as climate change, and swaying voters in political elections. Yellow press existed long before the todigital age but had limited reach when compared mainstream press. However, with the introduction of social media, news that are extremely biased (hyperpartisan) tend to spread more quickly than the ones that are not (Vosoughi et al., 2018). Therefore, there is a need for automatic detection methods of hyperpartisan news. Computational methods for fighting fake news mainly focus on automatic fact-checking rather than looking at writing styles of news articles (Potthast et al., 2018). SemEval-2019 Hyperpartisan News Detection shared task aims to study the ability of a system to detect if a given article exhibits a hyperpartisan argumentation writing style to convince readers of a certain position. The shared task introduces a binary classification task of classifying an article into one of two possibilities: mainstream or hyperpartisan. The data for the shared task was introduced by (Kiesel et al., 2019) and consists of 645 of articles from mainstream, left-wing, and right-wing publishers. The articles from both left-wing and right-wing publishers were labeled as hyperpartisan. The baseline system to detect hyperpartisan developed by (Potthast et al., 2018) uses Unmasking (Koppel et al., 2007) and was trained on 1,627 of articles. The articles are from nine publishers in the US: three mainstream (ABC News, CNN, and Politico), three left-wing (Addicting Info, Occupy Democrats, and The Other 98%), and three right-wing (Eagle Rising, Freedom Daily, and Right Wing News). Their model had a best accuracy of 75% by using stylistic features. However, their model is not directly comparable with ours since the dataset for the shared task is different. In the following sections, we describe our system for identifying hyperpartisan news articles as part …

Content truncated to 500 words