QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2804–2819, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2804 QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines Valentina Pyatkin1 Ayal Klein1 Reut Tsarfaty1,2 Ido Dagan1 1Computer Science Department, Bar Ilan University 2Allen Institute for Artificial Intelligence {valpyatkin,ayal.s.klein,ido.k.dagan,reut.tsarfaty}@gmail.com Abstract Discourse relations describe how two propo- sitions relate to one another, and identifying them automatically is an integral part of nat- ural language understanding. However, anno- tating discourse relations typically requires ex- pert annotators. Recently, different semantic aspects of a sentence have been represented and crowd-sourced via question-and-answer (QA) pairs. This paper proposes a novel rep- resentation of discourse relations as QA pairs, which in turn allows us to crowd-source wide- coverage data annotated with discourse rela- tions, via an intuitively appealing interface for composing such questions and answers. Based on our proposed representation, we col- lect a novel and wide-coverage QADiscourse dataset, and present baseline algorithms for predicting QADiscourse relations. 1 Introduction Relations between propositions are commonly termed Discourse Relations, and their importance to the automatic understanding of the content and structure of narratives has been extensively studied (Grosz and Sidner, 1986; Asher et al., 2003; Web- ber et al., 2012). The automatic parsing of such relations is thus relevant to multiple areas of NLP research, from extractive tasks such as document summarization to automatic analyses of narratives and event chains (Li et al., 2016; Lee and Gold- wasser, 2019). So far, discourse annotation has been mainly conducted by experts, relying on carefully crafted linguistic schemes. Two cases in point are PDTB (Prasad et al., 2008; Webber et al., 2019) and RST (Mann and Thompson, 1988; Carlson et al., 2002). Such annotation however is slow and costly. Crowd-sourcing discourse relations, instead of us- ing experts, can be very useful for obtaining larger- scale training data for discourse parsers. The executions were spurred by lawmakers requesting action to curb rising crime rates. What is the reason lawmakers requested action? to curb rising crime rates What is the result of lawmakers requesting action to curb rising crime rates? the executions were spurred I decided to do a press conference [...], and I did that going into it knowing there would be consequences. Despite what did I decide to do a press conference? know- ing there would be consequences Table 1: Sentences with their corresponding Question- and-Answer pairs. The bottom example shows how im- plicit relations are captured as QAs. One plausible way for acquiring linguistically meaningful annotations from laymen is using the relatively recent QA methodology, that is, convert- ing a set of linguistic concepts to intuitively simple Question-and-Answer (QA) pairs. Indeed, casting the semantic annotations of individual propositions as narrating a QA pair gained increasing attention in recent years, ranging from QA driven Semantic Role Labeling (QASRL) (He et al., 2015; FitzGer- ald et al., 2018; Roit et al., 2020) to covering all semantic relations as in QAMR (Michael et al., 2018). These representations were …

Content truncated to 500 words