Team QCRI-MIT at SemEval-2019 Task 4: Propaganda Analysis Meets Hyperpartisan News Detection

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1041–1046 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1041 Team QCRI-MIT at SemEval-2019 Task 4: Propaganda Analysis Meets Hyperpartisan News Detection Abdelrhman Saleh1, Ramy Baly2, Alberto Barr´on-Cede˜no3, Giovanni Da San Martino3, Mitra Mohtarami2, Preslav Nakov3, James Glass2 1Harvard University, MA, USA 2MIT Computer Science and Artificial Intelligence Laboratory, MA, USA 3Qatar Computing Research Institute, HBKU, Qatar abdelrhman saleh@college.harvard.edu, {baly, mitram, glass}@mit.edu {albarron, gmartino, pnakov}@hbku.edu.qa Abstract We describe our submission to SemEval-2019 Task 4 on Hyperpartisan News Detection. We rely on a variety of engineered features origi- nally used to detect propaganda. This is based on the assumption that biased messages are propagandistic and promote a particular po- litical cause or viewpoint. In particular, we trained a logistic regression model with fea- tures ranging from simple bag of words to vo- cabulary richness and text readability. Our system achieved 72.9% accuracy on the man- ually annotated testset, and 60.8% on the test data that was obtained with distant supervi- sion. Additional experiments showed that sig- nificant performance gains can be achieved with better feature pre-processing.1 1 Introduction The rise of social media has enabled people to eas- ily share information with a large audience with- out regulations or quality control. This has al- lowed malicious users to spread disinformation and misinformation (a.k.a. “fake news”) at an un- precedented rate. Fake news is typically charac- terized as being hyperpartisan (one-sided), emo- tional and riddled with lies (Potthast et al., 2018). The SemEval-2019 Task 4 on Hyperpartisan News Detection (Kiesel et al., 2019) focused on the chal- lenge of automatically identifying whether a text is hyperpartisan or not. While hyperpartisanship is defined as “exhibit- ing one or more of blind, prejudiced, or unrea- soning allegiance to one party, faction, cause, or person”, we model this task as a binary document classification problem. Scholars have argued that all biased messages can be considered propagan- distic, regardless of whether the bias was inten- tional or not (Ellul, 1965, p. XV). 1Our system is available at https://github.com/ AbdulSaleh/QCRI-MIT-SemEval2019-Task4 Thus, we approached the task departing from an existing model for propaganda identification (Barr´on-Cede˜no et al., 2019). Our hypothesis is that propaganda is inherent in hyperpartisanship and that the two problems are two sides of the same coin, and thus solving one of them would help solve the other. Our system consists of a lo- gistic regression model that is trained with a va- riety of engineered features that range from word and character TF.IDF n-grams and lexicon-based features to more sophisticated features that repre- sent different aspects of the article’s text such vo- cabulary richness and language complexity. Our official submission achieved an accuracy of 72.9% (while the winning system achieved 82.2%). This was achieved using word and char- acter n-grams. Moreover, post-submission ex- periments have shown that further performance improvements can be achieved by carefully pre- processing the engineered features. 2 Related Work The analysis of bias and disinformation has at- …

Content truncated to 500 words