Brenda Starr at SemEval-2019 Task 4: Hyperpartisan News Detection

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 924–928 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 924 Brenda Starr at SemEval-2019 Task 4: Hyperpartisan News Detection Olga Papadopoulou, Giorgos Kordopatis-Zilos, Markos Zampoglou, Symeon Papadopoulos, Yiannis Kompatsiaris Centre for Research and Technology Hellas, Information Technologies Institute, Thessaloniki, Greece (olgapapa,georgekordopatis,markzampoglou,papadop,ikom)@iti.gr Abstract In the effort to tackle the challenge of Hyper- partisan News Detection, i.e., the task of de- ciding whether a news article is biased towards one party, faction, cause, or person, we experi- mented with two systems: i) a standard super- vised learning approach using superficial text and bag-of-words features from the article title and body, and ii) a deep learning system com- prising a four-layer convolutional neural net- work and max-pooling layers after the embed- ding layer, feeding the consolidated features to a bi-directional recurrent neural network. We achieved an F-score of 0.712 with our best ap- proach, which corresponds to the mid-range of performance levels in the leaderboard. 1 Introduction The emerging issue of online disinformation has lately attracted the public attention and is per- ceived as a major risk for democracy and society. Media content (text, images, videos) is often dis- seminated on the Internet with the purpose of ma- nipulating public opinion. Hyperpartisan news de- tection is a problem arising as a result of the inten- tion of publishers to influence readers in favour of a given party, idea or person. The SemEval 2019 Task 4 (Kiesel et al., 2019) seeks solutions to this challenge, in particular text-based approaches that can detect hyperpartisan news articles. We experimented with two approaches: i) a standard supervised learning approach using su- perficial text and bag-of-words features, and ii) a deep learning system. We deployed the devel- oped systems on TIRA (Potthast et al., 2019) (a platform that supports software submissions) and its evaluation was conducted on unseen news ar- ticles. The results of our submissions, which are presented in Table 1, are promising, yet there is still considerable room for improvement. Our best resulting approach was the deep learning sys- tem, which scored an F-score of 0.712. The im- plemented approaches are described below along with additional experiments that were conducted on the provided training and validation datasets. 2 Data The dataset provided by the organizers of the task (Kiesel et al., 2019) consists of news arti- cles, half of which are labelled as hyperpartisan. It is split into two sets, the training and the vali- dation set, where for each article the article title, body and published date are provided. The train- ing set consists of 500.000 news articles and it is used as training set for the presented experiments and the provided validation set (150.000 news ar- ticles) is used for validating the approaches. A small dataset of 645 news articles, manually anno- tated, is also provided but not used in the following experiments neither as training nor as validation data. For the evaluation phase, two small datasets of 628 and …

Content truncated to 500 words