Seeing Things from a Different Angle:Discovering Diverse Perspectives about Claims

Proceedings of NAACL-HLT 2019, pages 542–557 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 542 Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch, Dan Roth Department of Computer and Information Science, University of Pennsylvania {sihaoc,danielkh,wenpeng,ccb,danroth}@cis.upenn.edu Abstract One key consequence of the information revo- lution is a significant increase and a contami- nation of our information supply. The practice of fact-checking won’t suffice to eliminate the biases in text data we observe, as the degree of factuality alone does not determine whether biases exist in the spectrum of opinions visi- ble to us. To better understand controversial issues, one needs to view them from a diverse yet comprehensive set of perspectives. For example, there are many ways to respond to a claim such as “animals should have law- ful rights”, and these responses form a spec- trum of perspectives, each with a stance rel- ative to this claim and, ideally, with evidence supporting it. Inherently, this is a natural lan- guage understanding task, and we propose to address it as such. Specifically, we propose the task of substantiated perspective discov- ery where, given a claim, a system is expected to discover a diverse set of well-corroborated perspectives that take a stance with respect to the claim. Each perspective should be substan- tiated by evidence paragraphs which summa- rize pertinent results and facts. We construct PERSPECTRUM, a dataset of claims, perspectives and evidence, making use of online debate websites to create the ini- tial data collection, and augmenting it using search engines in order to expand and diver- sify our dataset. We use crowdsourcing to filter out noise and ensure high-quality data. Our dataset contains 1k claims, accompanied by pools of 10k and 8k perspective sentences and evidence paragraphs, respectively. We provide a thorough analysis of the dataset to highlight key underlying language understand- ing challenges, and show that human baselines across multiple subtasks far outperform ma- chine baselines built upon state-of-the-art NLP techniques. This poses a challenge and an op- portunity for the NLP community to address. Figure 1: Given a claim, a hypothetical system is ex- pected to discover various perspectives that are sub- stantiated with evidence and their stance with respect to the claim. 1 Introduction Understanding most nontrivial claims requires in- sights from various perspectives. Today, we make use of search engines or recommendation systems to retrieve information relevant to a claim, but this process carries multiple forms of bias. In particu- lar, they are optimized relative to the claim (query) presented, and the popularity of the relevant doc- uments returned, rather than with respect to the diversity of the perspectives presented in them or whether they are supported by evidence. In this paper, we explore an approach to miti- gating this selection bias (Heckman, 1979) when studying (disputed) claims. Consider the claim shown in Figure 1: “animals should have lawful rights.” One might compare the biological simi- larities/differences between humans and …

Content truncated to 500 words