Automatic Explanation Generation For Climate Science Claims

Automatic Explanation Generation For Climate Science Claims Rui Xing1 Shraey Bhatia1 Timothy Baldwin1,2 Jey Han Lau1 1The University of Melbourne 2MBZUAI ruixing@student.unimelb.edu.au, shraeybhatia@gmail.com, tb@ldwin.net, jeyhan.lau@gmail.com Abstract Climate change is an existential threat to hu- manity, the proliferation of unsubstantiated claims relating to climate science is manipulat- ing public perception, motivating the need for fact-checking in climate science. In this work, we draw on recent work that uses retrieval- augmented generation for veracity prediction and explanation generation, in framing expla- nation generation as a query-focused multi- document summarization task. We adapt PRIMERA to the climate science domain by adding additional global attention on claims. Through automatic evaluation and qualitative analysis, we demonstrate that our method is effective at generating explanations. 1 Introduction The rapid dissemination of misinformation and disinformation through social media is a press- ing issue, especially in the domain of climate sci- ence (Diggelmann et al., 2020; Anderegg et al., 2010) where climate change has become one of the biggest challenges to humankind. Claims such as 97% consensus on human-caused global warming has been disproven seed scepticism, discredit cli- mate science, and manipulate public perception and interpretation. To alleviate the influence of such potentially false claims, experts have increasingly engaged in science communication, including in- vestigating such claims based on scientific evidence through websites such as climatefeedback.org and skepticalscience.com. This paper concerns the use of external knowledge to semi-automate the process of claim verification, as an assistive technology for contributors to such websites. Inspired by recent work on retrieval-augmented generation (Lewis et al., 2020) and explainable fact-checking (Atanasova et al., 2020), we aim to (semi-)automate the process of claim veracity clas- sification along with explanation generation. Our work draws on previous work on generating expla- nations in the climate science domain (Bhatia et al., Text Label C: Sea-level rise is not accelerating. REFU E1: Climate-change driven accelerated sea- level rise detected in the altimeter era. REFU E2: Antarctica ice melt has accelerated by 280% in the last 4 decades. REFU E3: However scientists have found that ice is being lost, and at an accelerating rate. REFU E4: Climate scientists expect the rate to further accelerate during the 21st century. NO INFO E5: More precise data gathered from satellite radar measurements reveal an accelerating rise of 7.5cm (3.0in) from 1993 to 2017, which is a trend of roughly 30cm (12in) per century. NO INFO Table 1: An example claim (“C”) and associated evi- dence passages (“Ek”) from Climate-Fever (“REFU” = REFUTES; “NO INFO” = NOT ENOUGH INFO). 2021a) in using claims to retrieve relevant docu- ments from knowledge sources and then generate explanations based on these documents. Unlike prior work, we frame it as a query-focused sum- marization task (Moll´a et al., 2020; Sarker et al., 2013), where the query is a claim in our case, and the goal is to summarize information from the re- trieved documents that addresses the claim. We evaluate our framework quantitatively and qualita- tively, and explore the impact of different variants of attention on …

Content truncated to 500 words