Measuring Correlation-to-Causation Exaggeration in Press Releases

Proceedings of the 28th International Conference on Computational Linguistics, pages 4860–4872 Barcelona, Spain (Online), December 8-13, 2020 4860 Measuring Correlation-to-Causation Exaggeration in Press Releases Bei Yu1, Jun Wang2, Lu Guo3, and Yingya Li1 1School of Information Studies, Syracuse University 2Independent Researcher, Syracuse, NY 3School of Public Administration, Sichuan University {byu,yli48}@syr.edu, {junwang4,luguo2019}@gmail.com Abstract Press releases have an increasingly strong influence on media coverage of health research; how- ever, they have been found to contain seriously exaggerated claims that can misinform the public and undermine public trust in science. In this study we propose an NLP approach to identify ex- aggerated causal claims made in health press releases that report on observational studies, which are designed to establish correlational findings, but are often exaggerated as causal. We devel- oped a new corpus and trained models that can identify causal claims in the main statements in a press release. By comparing the claims made in a press release with the corresponding claims in the original research paper, we found that 22% of press releases made exaggerated causal claims from correlational findings in observational studies. Furthermore, universities exagger- ated more often than journal publishers by a ratio of 1.5 to 1. Encouragingly, the exaggeration rate has slightly decreased over the past 10 years, despite the increase of the total number of press releases. More research is needed to understand the cause of the decreasing pattern. 1 Introduction Recent studies have found that press releases issued by research institutions are a major source of exag- geration in science communication, which is later spread to mainstream media (Woloshin and Schwartz, 2002; Brechman et al., 2009; Sumner et al., 2014; Sumner et al., 2016). Exaggeration in press releases threatens to undermine public trust in science, which is a foundation for the scientific research enterprise. The influence of press releases on science communication has increased since the 1980s. With the growing competition for reputation and funding, research institutions are increasingly using press re- leases as a public relations tool for research promotion (Brechman et al., 2009; Sumner et al., 2014). At the same time, independent journalism faces financial challenges and staff shortage (Galewitz, 2006; Schwitzer, 2008). Newspapers, especially small newspapers, rely heavily on press release material to write science news stories (De Semir et al., 1998; Schwitzer, 2008; Woloshin et al., 2009; Taylor et al., 2015). Prior research also shows that high quality press releases seem to improve the quality of associated news stories (Schwartz et al., 2012). Since press releases are now the dominant link between academia and news media, the information quality of press releases plays a critical role in communicating science research to the public. The problem of exaggeration in press releases is particularly severe in health research. For example, a manual analysis of a sample of UK press releases in health research found that over a third of them contained exaggerated advice, causal claims from correlational findings, or inference of animal studies to humans (Sumner et al., 2014). The errors and inaccuracies in reporting …

Content truncated to 500 words