Harvey Mudd College at SemEval-2019 Task 4: The Carl Kolchak Hyperpartisan News Detector

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 957–961 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 957 Harvey Mudd College at SemEval-2019 Task 4: The Carl Kolchak Hyperpartisan News Detector Celena Chen cechen@hmc.edu Celine Park cpark@hmc.edu Jason Dwyer jdwyer@hmc.edu Julie Medero jmedero@hmc.edu Abstract We use various natural processing and ma- chine learning methods to perform the Hyper- partisan News Detection task. In particular, some of the features we look at are bag-of- words features , the title’s length, number of capitalized words in the title, and the sentiment of the sentences and the title. By adding these features, we see improvements in our evalua- tion metrics compared to the baseline values. We find that sentiment analysis helps improve our evaluation metrics. We do not see a bene- fit from feature selection. Overall, our system achieves an accuracy of 0.739, finishing 18th out of 42 submissions to the task. From our work, it is evident that both title features and sentiment of articles are meaningful to the hy- perpartisanship of news articles. 1 Introduction In 2019, Task 4 of the SemEval Workshop asked participants to automatically identify hyperparti- san texts (Kiesel et al., 2019). Hyperpartisan news detection is the problem of building a classifier us- ing natural language processing techniques in or- der to label news articles as either hyperpartisan or neutral in content bias. Hyperpartisan articles, in this case, can be defined as articles which are very polarized and extremely biased towards one polit- ical party. This task is quite relevant in today’s po- litical climate, with reports of ”fake news” articles heavily influencing votes and people’s support of some candidates running for government offices. The issue is especially egregious because these bi- ased news articles are informing political opinions of people who believe them to be factual and im- partial, with no easy way to remove or detect hy- perpartisan news articles. This is also a non-trivial task, as hyperpartisan news is not extremely ex- plicit in its bias, and even human readers do not always agree on which articles should be classified as hyperpartisan or what about those articles mer- its such a classification. There is no one unifying feature of hyperpartisan news, and even news pub- lishers which do produce hyperpartisan news are not guaranteed to only publish hyperpartisan news. Therefore, each article must be evaluated for its degree of hyperpartisanship, along many different axes of measurements. A functional and accurate hyperpartisan news detector would be useful for social media sites and other carriers of news to make sure that their news content is unbiased, and to be able to detect and perhaps remove or block sources of hyperparti- san news. Facebook, for example, has been un- der great public scrutiny due to the quantity and popularity of hyperpartisan news on its site. Hy- perpartisan news detection could also be useful for researchers seeking to understand the scope and impact of hyperpartisan news on the 2016 presi- dential election and …

Content truncated to 500 words