Team Howard Beale at SemEval-2019 Task 4: Hyperpartisan News Detection with BERT

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1007–1011 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1007 Team Howard Beale at SemEval-2019 Task 4: Hyperpartisan News Detection with BERT Osman Mutlu∗ Koc¸ University ˙Istanbul, Sarıyer omutlu@ku.edu.tr Ozan Arkan Can∗ Koc¸ University ˙Istanbul, Sarıyer ocan13@ku.edu.tr Erenay Dayanık University of Stuttgart Stuttgart erenaydayanik@gmail.com Abstract This paper describes our system for SemEval- 2019 Task 4: Hyperpartisan News Detection (Kiesel et al., 2019). We use pretrained BERT (Devlin et al., 2018) architecture and investi- gate the effect of different fine tuning regimes on the final classification task. We show that additional pretraining on news domain im- proves the performance on the Hyperpartisan News Detection task. Our system1 ranked 8th out of 42 teams with 78.3% accuracy on the held-out test dataset. 1 Introduction With the rapid spread of the Internet and next- generation media development, people started to follow news through the Internet by abandoning de facto sources such as television and radio. Re- cent studies reveal that 43% of Americans report often getting news online (Shearer and Gottfried, 2017). In parallel with that, there also has been a massive improvement in the NLP research in news domain to keep the content true, fair and unbiased. SemEval-2019 Task 4: Hyperpartisan News Detection, is yet another attempt under this objective. Hyperpartisan is defined as being ex- tremely biased in favor of a political party (Bastos and Mercea, 2017) and the aim of the shared task is to detect hyperpartisan argumentation in news text. Though it is an important task by itself, hy- perpartisan argument detection is also considered as a very first step (or even replacement) of fake news detection, because it has been shown by (Pot- thast et al., 2018) that there is a high positive cor- relation between having a hyperpartisan argumen- tation and being fake for news items. In this shared task, we seek to model this prob- lem as a text classification task. In general, the ∗equal contribution 1https://github.com/ozanarkancan/hyperpartisan task aims to label the text in the question with one or more classes or categories. The main ques- tion of text classification is how to mathematically represent the words/tokens such that they retain their original meaning in the context they appear. This question has been tried to be answered in many different ways so far. In earlier work, peo- ple mainly used the ”bag of words” approach in algorithms such as Naive Bayes, Decision Tree, and SVM. Then, (Mikolov et al., 2013) advanced the field further by introducing word embeddings, capturing a somewhat meaningful representation of words. However, recent studies (Peters et al., 2018; Radford et al., 2018; Devlin et al., 2018) showed that contextual word embeddings perform quite better than traditional word embeddings in many different NLP tasks as a result of their supe- rior capacity of meaning representation. Among those, BERT attracts researchers most because of (i) its transformer based architecture enabling faster training and (ii) state of the art results in …

Content truncated to 500 words