Harvey Mudd College at SemEval-2019 Task 4: The D.X. Beaumont Hyperpartisan News Detector

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 967–970 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 967 1 Harvey Mudd College at SemEval-2019 Task 4: The D.X. Beaumont Hyperpartisan News Detector Evan Amason Harvey Mudd College 301 Platt Boulevard Claremont, CA 91711 eamason@hmc.edu Jake Palanker Harvey Mudd College 301 Platt Boulevard Claremont, CA 91711 jpalanker@hmc.edu Mary Clare Shen Harvey Mudd College 301 Platt Boulevard Claremont, CA 91711 mshen@hmc.edu Julie Medero Harvey Mudd College 301 Platt Boulevard Claremont, CA 91711 jmedero@hmc.edu Abstract We use the 600 hand-labelled articles from Se- mEval Task 4 (Kiesel et al., 2019) to hand- tune a classifier with 3000 features for the Hy- perpartisan News Detection task. Our final system uses features based on bag-of-words (BoW), analysis of the article title, language complexity, and simple sentiment analysis in a naive Bayes classifier. We trained our fi- nal system on the 600,000 articles labelled by publisher. Our final system has an accuracy of 0.653 on the hand-labeled test set. The most effective features are the Automated Readabil- ity Index and the presence of certain words in the title. This suggests that hyperpartisan writ- ing uses a distinct writing style, especially in the title. 1 Introduction Hyperpartisan news is becoming more mainstream as online sources gain popularity. Hyperpartisan news is news written from an extremely partisan perspective, such that the goal is reinforcing exist- ing belief structures in the party’s ideology rather than conveying facts. Such hyperpartisan writing tends to amplify political divisions and increase animosity between opposing political ideologies. Hyperpartisan news sources also output fake news at startling rates (Silverman et al., 2016). Auto- matic detection of fake news is difficult, but de- tecting hyperpartisan news can help, and it can also expose biases in journalism. This task is challenging to automate because it is even diffi- cult for humans: fake and biased news articles get shared on social media at high rates, and even labels that were hand-generated by professionals have errors (Silverman et al., 2016). We attempt to use various features of political news articles to train a multinomial Naive Bayes classifier to complete this task. We use a set of bag-of-words (BoW) features for words appearing in the title of each article, and for words appearing in the arti- cle text. With these features, we identified a set of words that characterize hyperpartisan writing. We also considered complexity features such as type-to-token ratio and automated readability in- dex. Based on the performance of these features we attempt to answer the question of whether hy- perpartisan writing is more or less complex than non-hyperpartisan writing. A successful classifier could be very useful in today’s society. For exam- ple, it could be used to create a browser plug-in to check online articles for political bias in real time as the user reads. People on social media could use it to verify the legitimacy of a political article before sharing it with their followers. En- couraging people to share factual …

Content truncated to 500 words