Team Fernando-Pessa at SemEval-2019 Task 4: Back to Basics in Hyperpartisan News Detection

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 999–1003 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 999 Team Fernando-Pessa at SemEval-2019 Task 4: Back to Basics in Hyperpartisan News Detection Andr´e Ferreira Cruz Universidade do Porto Faculdade de Engenharia andre.ferreira.cruz@fe.up.pt Gil Rocha Universidade do Porto Faculdade de Engenharia LIACC gil.rocha@fe.up.pt Rui Sousa-Silva Universidade do Porto Faculdade de Letras CLUP rssilva@letras.up.pt Henrique Lopes Cardoso Universidade do Porto Faculdade de Engenharia LIACC hlc@fe.up.pt Abstract This paper describes our submission1 to the SemEval 2019 Hyperpartisan News Detection task. Our system aims for a linguistics-based document classification from a minimal set of interpretable features, while maintaining good performance. To this goal, we follow a feature-based approach and perform sev- eral experiments with different machine learn- ing classifiers. On the main task, our model achieved an accuracy of 71.7%, which was improved after the task’s end to 72.9%. We also participate in the meta-learning sub-task, for classifying documents with the binary clas- sifications of all submitted systems as input, achieving an accuracy of 89.9%. 1 Introduction Hyperpartisan news detection consists in identify- ing news that exhibit extreme bias towards a sin- gle side (Potthast et al., 2018). The shift, in news consumption behavior, from traditional outlets to social media platforms has been accompanied by a surge of fake and/or hyperpartisan news articles in recent years (Gottfried and Shearer, 2017), rais- ing concerns in both researchers and the general public. As ideologically aligned humans prefer to believe in ideologically aligned news (Allcott and Gentzkow, 2017), these tend to be shared more of- ten and, thus, spread at a fast and unchecked pace. Moreover, there is a large intersection of ‘fake’ and ‘hyperpartisan’ news, as 97% of fake news ar- ticles in BuzzFeed’s Facebook fact-check dataset are also hyperpartisan (Silverman et al., 2016). However, the detection/classification and con- sequent regulation of online content must be done 1https://github.com/AndreFCruz/ semeval2019-hyperpartisan-news with careful consideration, as any automatic sys- tem risks unintended censorship (Akdeniz, 2010). As such, we aim for a linguistically-guided model from a set of interpretable features, together with classifiers that facilitate inspection of what the model has learned, such as Random Forests (Ho, 1995), Support Vector Machines (Cortes and Vap- nik, 1995) and Gradient Boosted Trees (Drucker and Cortes, 1996). Neural network models are left out for their typically less self-explanatory nature. The SemEval 2019 Task 4 (Kiesel et al., 2019) challenged participants to build a system for hy- perpartisan news detection. The provided dataset consists of 645 manually annotated articles (by- article dataset), as well as 750,000 articles auto- matically annotated publisher-wise (by-publisher dataset, split 80% for training and 20% for val- idation). Systems are ranked by accuracy on a set of unpublished test articles (from the by-article dataset), which has no publishers in common with the provided train dataset, preventing accuracy gains by profiling publishers. All experiments on this paper are performed on the gold-standard (by- article) corpus, as this was the official dataset. The rest of the paper …

Content truncated to 500 words