Team Bertha von Suttner at SemEval-2019 Task 4: Hyperpartisan News Detection using ELMo Sentence Representation Convolutional Network

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 840–844 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 840 Team Bertha von Suttner at SemEval-2019 Task 4: Hyperpartisan News Detection using ELMo Sentence Representation Convolutional Network Ye Jiang, Johann Petrak, Xingyi Song, Kalina Bontcheva, Diana Maynard Department of Computer Science University of Sheffield Sheffield , UK {yjiang18,johann.petrak,x.song, k.bontcheva,d.maynard}@sheffield.ac.uk Abstract This paper describes the participation of team “bertha-von-suttner” in the SemEval2019 task 4 Hyperpartisan News Detection task. Our system1 uses sentence representations from averaged word embeddings generated from the pre-trained ELMo model with Convolutional Neural Networks and Batch Normalization for predicting hyperpartisan news. The final pre- dictions were generated from the averaged pre- dictions of an ensemble of models. With this architecture, our system ranked in first place, based on accuracy, the official scoring metric. 1 Introduction Hyperpartisan news is typically defined as news which exhibits an extremely biased opinion in favour of one side, or unreasoning allegiance to one party (Potthast et al., 2017). SemEval- 2019 Task 4 on “Hyperpartisan News Detection” (Kiesel et al., 2019) is a document-level classifica- tion task which requires building a precise and re- liable algorithm to automatically discriminate hy- perpartisan news from more balanced stories. One of the major challenges of this task is that the model must have the ability to adapt to a large range of article sizes. In one of the training data sets, the by-publisher corpus, the average article length is 796 tokens, but the longest document has 93,714 tokens. Most state-of-the-art neural net- work approaches for document classification use a token sequence as network input (Kim, 2014; Yin and Sch¨utze, 2016; Conneau et al., 2016). This implies either a high computational cost when a very large maximum sequence length is used to fully represent the longest articles, or alternatively potentially a significant loss of information if the 1The code is available at https://github.com/GateNLP/semeval2019- hyperpartisan-bertha-von-suttner sequence length is restricted to a manageable num- ber of initial tokens from the document. In this paper, we introduce the ELMo Sentence Representation Convolutional (ESRC) Network. We first pre-calculate sentence level embeddings as the average of ELMo (Peters et al., 2018) word embeddings for each sentence, and represent the document as a sequence of such sentence embed- dings. We then apply a lightweight convolutional Neural Network (CNN), along with Batch Nor- malization (BN), to learn the document represen- tations and predict the hyperpartisan classification. Two types of data set have been made avail- able for the task. The by-publisher corpus con- tains 750K articles which were automatically clas- sified based on a categorization of the political bias of the news source. This dataset was split into a training set of 600K articles and a validation set of 150K articles, where all the articles in the valida- tion set originated from sources not in the training set. The second set, by-article, contains just 645 articles which were labelled manually. The final evaluation (Potthast et al., 2019) was carried out on …

Content truncated to 500 words