Cardiff University at SemEval-2019 Task 4: Linguistic Features for Hyperpartisan News Detection

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 929–933 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 929 Cardiff University at SemEval-2019 Task 4: Linguistic Features for Hyperpartisan News Detection Carla P´erez-Almendros, Luis Espinosa-Anke and Steven Schockaert School of Computer Science and Informatics Cardiff University, UK {perezalmendrosc,espinosa-ankel,schockaerts1}@cardiff.ac.uk Abstract This paper summarizes our contribution to the Hyperpartisan News Detection task in Se- mEval 2019. We experiment with two differ- ent approaches: 1) an SVM classifier based on word vector averages and hand-crafted lin- guistic features, and 2) a BiLSTM-based neu- ral text classifier trained on a filtered training set. Surprisingly, despite their different nature, both approaches achieve an accuracy of 0.74. The main focus of this paper is to further ana- lyze the remarkable fact that a simple feature- based approach can perform on par with mod- ern neural classifiers. We also highlight the ef- fectiveness of our filtering strategy for training the neural network on a large but noisy train- ing set. 1 Introduction In the era of misinformation, the challenge of dif- ferentiating reality from frames, facts from opin- ions, is becoming increasingly important. Con- cepts such as Fake News, Fact Checking or Post- Truth Era, generally unknown a few years ago (Lewandowsky et al., 2017), started to play an important part in media, academic papers and even in Natural Language Processing (NLP) tasks (Rashkin et al., 2017; Shu et al., 2017; Wang, 2017). Nowadays, strongly opinionated news sto- ries can offer biased information, as is the case with hyperpartisan articles. A text is considered hyperpartisan when it is highly polarized towards an extreme position. Potthast et al. (2018) ana- lyzed hyperpartisanism in relation to fake news, to discover that a very similar writing style could be associated both with right-wing and left-wing polarized stories. This shared style of biased arti- cles was different from that of mainstream articles. Task 4 in SemEval 2019 (Kiesel et al., 2019) con- sisted in a classification challenge where news ar- ticles had to be sorted out as hyperpartisan or non- hyperpartisan. Our approach addressed this chal- lenge via two different models, which included 1) an SVM classifier based on word embeddings averages and handcrafted linguistic features and 2) a recurrent BiLSTM neural network classifier trained on filtered data. The reason and process for filtering data will be explained in section 3.2. Word embeddings have remained central to the state-of-the-art approaches in NLP since the intro- duction of Word2Vec (Mikolov et al., 2013) and GloVe (Pennington et al., 2014) models. In ad- dition to modelling word meaning, embeddings can also be applied to longer units of signifi- cance, such as phrases, sentences, paragraphs or entire documents (Le and Mikolov, 2014). How- ever, although such vector representations often enable the best results in a variety of NLP chal- lenges, some tasks still benefit from linguisti- cally inspired approaches. In fact, recent work has proved how linguistic features and stylome- try could improve the performance of deep learn- ing …

Content truncated to 500 words