A Review of Dataset and Labeling Methods for Causality Extraction

Proceedings of the 28th International Conference on Computational Linguistics, pages 1519–1531 Barcelona, Spain (Online), December 8-13, 2020 1519 A Review of dataset and labeling methods for causality extraction Jinghang Xu1,2, Wanli Zuo1,2, Shining Liang1,2, Xianglin Zuo1,2∗ 1College of Computer Science and Technology, Jilin University 2Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education (xujh17, liangsn19)@mails.jlu.edu.cn zuowl@jlu.edu.cn, 295228473@qq.com Abstract Causality represents the most important kind of correlation between events. Extracting causal- ity from text has become a promising hot topic in NLP. However, there is no mature research systems, evaluation rules and datasets for public evaluation. Moreover, there is a lack of unified causal sequence labeling methods, which constitute the key factors that hinder the progress of causality extraction research. We survey the limitations and shortcomings of existing causality research field comprehensively from the aspects of basic concepts, extraction methods, exper- imental data, and labeling methods, so as to provide reference for future research on causality extraction. We summarize the existing causality datasets, explore their practicability and ex- tensibility from multiple perspectives. Aiming at the problem of causal sequence labeling, we analyze the existing methods of causal sequence labeling, with a summarizations of its regulation. Multiple candidate causal labeling sequences are put forward according to labeling controversy to explore the optimal labeling method through experiments, and suggestions are provided for selecting labeling method. 1 Introduction Causality represents a kind of relationships between “cause” and “effect”, stating that the happening of causes will trigger the happening of effects, which constitutes the basics for inference and reasoning. In early days, people found causal relations from production process and daily lives manually, slow and inaccurate. The ever growing web resources have made it feasible to automatically extracts causality from text, triggering an emerging and hot topic in NLP, with abundant downstream application tasks such as event detection and prediction (Radinsky et al., 2012), questions answering (Hashimoto et al., 2014). For example, the new coronary pneumonia has drawn global wide attention recently, the causal relationship between “new coronavirus” and “eating wild animals” can be inferred by an expert system with causal knowledge, and it may be predicted that “new coronary pneumonia” can lead to “death”. Artificial intelligence can be used to assist medical research and strengthen prevention and treatment during early stage of infection. Therefore, causal relation extraction is a basic task in text mining, which is driven by human’s instinctive desire for knowledge. Many researchers have devoted themselves to the research of causal relationship extraction, however, it is still a new field with some open problems to slove and publicly evaluated datasets. The existing relevant studies each has its own research system that cannot be compared horizontally, so the systematic research system is the key factors for the progress of causality extraction research. Targeting at the limitations and problems in causality research, we comprehensively review the basic concepts, extraction methods, experimental data, labeling methods from multiple perspectives, for future research on causality extraction as a reference. The main contributions of this …

Content truncated to 500 words