CausalQA: A Benchmark for Causal Question Answering

Proceedings of the 29th International Conference on Computational Linguistics, pages 3296–3308 October 12–17, 2022. 3296 CausalQA: A Benchmark for Causal Question Answering Alexander Bondarenko ⋆ Magdalena Wolska † Stefan Heindorf ‡ Lukas Blübaum ‡ Axel-Cyrille Ngonga Ngomo ‡ Benno Stein † Pavel Braslavski §,¶ Matthias Hagen ⋆ Martin Potthast ∥ ⋆Martin-Luther-Univeristät Halle-Wittenberg †Bauhaus-Universität Weimar ‡Paderborn University §Ural Federal University ¶HSE University ∥Leipzig University Abstract At least 5% of questions submitted to search engines ask about cause–effect relationships in some way. To support the development of tailored approaches that can answer such questions, we construct Webis-CausalQA-22, a benchmark corpus of 1.1 million causal ques- tions with answers. We distinguish different types of causal questions using a novel typol- ogy derived from a data-driven, manual anal- ysis of questions from ten large question an- swering (QA) datasets. Using high-precision lexical rules, we extract causal questions of each type from these datasets to create our cor- pus. As an initial baseline, the state-of-the-art QA model UnifiedQA achieves a ROUGE-L F1 score of 0.48 on our new benchmark. 1 Introduction The term “causality” usually refers to a directed relationship between events in which one is the cause of the occurrence of the other, called the ef- fect. Many empirical studies begin with a research question about a causal relationship, ranging from “yes/no”-questions such as “Does the quality of ed- ucation affect economic growth?” to open-ended questions such as “What causes depression?”. But the general public also frequently asks causal ques- tions. Figure 1 shows an example of the top Google, Bing, and Naver search result for the question “Can broccoli cause constipation?”. While Bing directly answers the question in the affirmative, Google’s featured snippet and Naver’s first snippet claim that broccoli actually has the opposite effect. With at most a few thousand question–answer pairs, existing causal question answering datasets are relatively small and include only one type of causal question, e.g., “yes/no”-questions (Hassan- zadeh et al., 2019; Kayesh et al., 2020), “what-if”- questions (Tandon et al., 2019), “why”-questions (Verberne et al., 2006a, 2008, 2010; Lal et al., 2021), or multiple-choice questions (Gordon et al., Can broccoli cause constipation? Foods that may help prevent constipation For many people, eating more high fiber foods can help ease constipation. These foods include: most vegetables, including carrots, peas, broccoli, and okra. https://www.medicalnewstoday.com › ... List of foods that can cause constipation and how to prevent it All Shopping Images News Videos Maps Yes According to 3 sources Since broccoli is so rich in fiber, too much of it can in fact cause diarrhea or even constipation. Fiber draws water into the GI tract, so if you aren’t drinking enough water then your GI tract may become dehydrated. This means the stools become hard and much more difficult to pass. Soluble fiber is the main culprit for this. Does Broccoli Make You Poop? SOLVED! - Health… healthbriefly.com Cruc broc gas unco expe eatin coul dige mea vege caus Can Con nutrit ALL IMAGES VIDEOS NEWS SHOPPING Can broccoli cause …

Content truncated to 500 words