Team yeon-zi at SemEval-2019 Task 4: Hyperpartisan News Detection by De-noising Weakly-labeled Data

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1052–1056 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1052 Team yeon-zi at SemEval-2019 Task 4: Hyperpartisan News Detection by De-noising Weakly-labeled Data Nayeon Lee∗, Zihan Liu∗, Pascale Fung Center for Artificial Intelligence Research (CAiRE) Department of Electronic and Computer Engineering Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong [nyleeaa, zliucr].connect.ust.hk, pascale@ece.ust.hk Abstract This paper describes our system submitted to SemEval-2019 Task 4: Hyperpartisan News Detection. We focus on removing the inher- ent noise in the hyperpartisanship dataset from both data-level and model-level by leveraging semi-supervised pseudo-labels and the state- of-the-art BERT model. Our model achieves 75.8% accuracy in the final by-article dataset without ensemble learning. 1 Introduction With the ever-growing usage of internet, the prob- lem of fake news that spreads in a destructive speed has attracted many attention. Fake news is a kind of news that is typically inflammatory, ex- tremely one-sided (hyper-partisan) or untruthful to mislead the public into having distorted belief. Previous works attempted to solve fake news problem from various aspects, ranging from knowledge-based (Wu et al., 2014; Shi and Weninger, 2016; Lee et al., 2018) to style- based (Wang, 2017; Potthast et al., 2018). There are some publicly available fake news datasets, however, often too small in size to be suitable for neural approaches (Horne and Adali, 2017; P´erez- Rosas et al., 2017). Recently, the organizers of SemEval2019 Task 4 (Kiesel et al., 2019) have re- leased large-scale dataset to address fake news de- tection as a hyper-partisan news detection prob- lem. The task is to determine whether a given arti- cle is hyper-partisan (extremely right-wing or left- wing) or not (mainstream). Such task will allow for pre-screening of semi-automatic fake news de- tection, and more importantly, bring us one step closer to solving fully automated fake news detec- tion. Initially, we focused on learning and utilizing useful features such as topic and sentiment infor- ∗ These two authors contributed equally. Figure 1: Illustration of filtering. Sample a and c are removed as pseudo-label ̸= by-publisher label. Sample b and d are removed as their prediction confidence was below threshold. mation. Considering the purpose of hyper-partisan news, we believed that the stance on politically sensitive topics would be crucial in determining hyperpartisanship. However, experiments showed that the dataset contains some inherent noise that acted as a big barrier to learning a good classi- fier: 1) noisy text inputs from an article that con- tain domain-specific (i.e. political) words, slangs and spelling mistakes which are likely to be out of vocabulary (OOV). 2) noisy labels that mainly re- sulted from using publisher-level information for labeling articles (i.e. all articles from left/right- wing publishers are labeled as “hyper-partisan”. For more detail, refer to Section 2). Nevertheless, human-labeled large-scale dataset creation is a very expensive and time-consuming task, thus, it is crucial to find a better way to uti- lize this weakly-labeled dataset. Therefore, we experimented with reducing noise …

Content truncated to 500 words