Fact vs. Opinion: the Role of Argumentation Features in News Classification

Proceedings of the 28th International Conference on Computational Linguistics, pages 6139–6149 Barcelona, Spain (Online), December 8-13, 2020 6139 Fact vs. Opinion: the Role of Argumentation Features in News Classification Tariq Alhindi† Smaranda Muresan†‡ Daniel Preot¸iuc-Pietro∗ †Department of Computer Science, Columbia University ‡Data Science Institute, Columbia University ∗Bloomberg {tariq, smara}@cs.columbia.edu dpreotiucpie@bloomberg.net Abstract A 2018 study led by the Media Insight Project showed that most journalists think that a clear marking of what is news reporting and what is commentary or opinion (e.g., editorial, op-ed) is essential for gaining public trust. We present an approach to classify news articles into news stories (i.e., reporting of factual information) and opinion pieces using models that aim to sup- plement the article content representation with argumentation features. Our hypothesis is that the nature of argumentative discourse is important in distinguishing between news stories and opinion articles. We show that argumentation features outperform linguistic features used previ- ously and improve on fine-tuned transformer-based models when tested on data from publishers unseen in training. Automatically flagging opinion pieces vs. news stories can aid applications such as fact-checking or event extraction. 1 Introduction Subjectivity in news reporting is rising in the recent years, especially in online-only publications (Blake and others, 2019). It was estimated that only 41% of publishers label their type of articles (e.g., editorial, review, analysis), and among those who label the types, there is a lack of consistency and clarity (Harris, 2017). A major finding of a 2018 study led by the Media Insight Project showed that most journalists (nearly 80%) think that their news organizations should clearly mark what is news reporting and what is opinion/commentary in order to combat fake news and gain public trust (The-Media-Insight-Project, 2018). Broadly, there are two types of news articles: 1) opinion articles written to present the opinion of the editor or board and aimed to persuade the readers with respect to a particular point of view, and 2) news stories, which aim to report factual news or events. Given that the intent of opinion articles is persuasion, we hypothesize that one of the key difference between news stories and opinion articles rests in the discourse structure and, in particular, the argumentative and persuasive aspects of the article. Figure 1 shows an example of a news story and an opinion article with two coarse-grained types of argumentative components highlighted (i.e., claims and premises). We can see that claims are more prevalent in the opinion article, while the news story contains more premises to support a small amount of claims. We study the predictive power of such coarse-grained argumentation features (claims and premises) for the task of news articles classification into news stories and opinion pieces. For short, we will refer to this binary task as news vs. opinion classification. To train our sentence-level argument component classification model (claim, premises, none), we use the corpus of editorial news labeled with argu- mentation strategies introduced by Al Khatib et al. (2016). We compare our approach that uses argu- mentation features …

Content truncated to 500 words