Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2931–2937 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking Hannah Rashkin† Eunsol Choi† Jin Yea Jang‡ Svitlana Volkova‡ Yejin Choi† †Paul G. Allen School of Computer Science & Engineering, University of Washington {hrashkin,eunsol,yejin}@cs.washington.edu ‡Data Sciences and Analytics, Pacific Northwest National Laboratory {jinyea.jang,svitlana.volkova}@pnnl.gov Abstract We present an analytic study on the lan- guage of news media in the context of po- litical fact-checking and fake news detec- tion. We compare the language of real news with that of satire, hoaxes, and pro- paganda to find linguistic characteristics of untrustworthy text. To probe the feasi- bility of automatic political fact-checking, we also present a case study based on PolitiFact.com using their factuality judg- ments on a 6-point scale. Experiments show that while media fact-checking re- mains to be an open research question, stylistic cues can help determine the truth- fulness of text. 1 Introduction Words in news media and political discourse have a considerable power in shaping people’s beliefs and opinions. As a result, their truthfulness is of- ten compromised to maximize impact. Recently, fake news has captured worldwide interest, and the number of organized efforts dedicated solely to fact-checking has almost tripled since 2014.1 Organizations, such as PolitiFact.com, actively in- vestigate and rate the veracity of comments made by public figures, journalists, and organizations. Figure 1 shows example quotes rated for truth- fulness by PolitiFact. Per their analysis, one com- ponent of the two statements’ ratings is the mis- leading phrasing (bolded in green in the figure). For instance, in the first example, the statement is true as stated, though only because the speaker hedged their meaning with the quantifier just. In the second example, two correlated events – Brexit 1https://www.poynter.org/2017/there-are-now-114-fact- checking-initiatives-in-47-countries/450477/ “You cannot get ebola from just riding on a plane or a bus.” Mostly True True False -Rated Mostly True by PolitiFact, (Oct. 2014) “Google search spike suggests many people don’t know why they voted for Brexit.” Mostly False True False -Rated Mostly False by PolitiFact, (June 2016) “There are already more American jobs in the solar industry than in coal mining.” True False -Rated True by PolitiFact, (May 2014) “By declaring that Pluto was no longer a planet, the (International Astronomical Union) put into place a planetary definition that would have even declassified Earth as a planet if it existed as far from the sun as Pluto does.” Half True True False -Rated Half True by PunditFact, (July 2015) Figure 1: Example statements rated by PolitiFact as mostly true and mostly false. Misleading phras- ing - bolded in green - was one reason for the in- between ratings. and Google search trends – are presented ambigu- ously as if they were directly linked. Importantly, like above examples, most fact- checked statements on PolitiFact are rated as nei- ther entirely true nor entirely false. Analysis in- dicates that falsehoods often arise from …

Content truncated to 500 words