The Causal News Corpus: Annotating Causal Relations in Event Sentences from News

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2298–2310 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2298 The Causal News Corpus: Annotating Causal Relations in Event Sentences from News Fiona Anting Tan1, Ali Hürriyeto˘glu2, Tommaso Caselli3, Nelleke Oostdijk4, Tadashi Nomoto5, Hansi Hettiarachchi6, Iqra Ameer7, Onur Uca8, Farhana Ferdousi Liza9, Tiancheng Hu10 1 Institute of Data Science, National University of Singapore, Singapore, 2 Koc University, Turkey, 3 Rijksuniversiteit Groningen, Netherlands, 4 Radboud University, Netherlands, 5 National Institute of Japanese Literature, Japan, 6 Birmingham City University, United Kingdom, 7 Centro de Investigación en Computación, Instituto Politécnico Nacional, Mexico, 8 Department of Sociology, Mersin University, Turkey, 9 University of East Anglia, United Kingdom, 10 ETH Zürich, Switzerland tan.f@u.nus.edu, ahurriyetoglu@ku.edu.tr, t.caselli@rug.nl, nelleke.oostdijk@ru.nl, nomoto@acm.org, hansi.hettiarachchi@mail.bcu.ac.uk, iqra@nlp.cic.ipn.mx, onuruca@mersin.edu.tr, f.liza@uea.ac.uk, tianhu@ethz.ch Abstract Despite the importance of understanding causality, corpora addressing causal relations are limited. There is a discrepancy between existing annotation guidelines of event causality and conventional causality corpora that focus more on linguistics. Many guidelines restrict themselves to include only explicit relations or clause-based arguments. Therefore, we propose an annotation schema for event causality that addresses these concerns. We annotated 3,559 event sentences from protest event news with labels on whether it contains causal relations or not. Our corpus is known as the Causal News Corpus (CNC). A neural network built upon a state-of-the-art pre-trained language model performed well with 81.20% F1 score on test set, and 83.46% in 5-folds cross-validation. CNC is transferable across two external corpora: CausalTimeBank (CTB) and Penn Discourse Treebank (PDTB). Leveraging each of these external datasets for training, we achieved up to approximately 64% F1 on the CNC test set without additional fine-tuning. CNC also served as an effective training and pre-training dataset for the two external corpora. Lastly, we demonstrate the difficulty of our task to the layman in a crowd-sourced annotation exercise. Our annotated corpus is publicly available, providing a valuable resource for causal text mining researchers. Keywords: causality, event causality, text mining, natural language understanding 1. Introduction Causality is a core cognitive concept and appears in many natural language processing (NLP) works that aim to tackle inference and understanding (Jo et al., 2021; Dunietz et al., 2020; Feder et al., 2021). Gen- erally, a causal relation is a semantic relationship be- tween two arguments known as cause and effect, in which the occurrence of one (cause argument) causes the occurrence of the other (effect argument) (Barik et al., 2016). Figure 1 depicts examples of sentences expressing causality, and ones that do not. Notice that causality can be expressed in various ways: The first causal ex- ample is signaled by the explicit causal marker “due to” while the second example has causality indicated by alternative lexicalizations such as “created”. In the last example, an implicit causal relation exists between the verbless clause and the matrix clause, conveying that the dissatisfaction with the package leads to the workers’ sit-in. For sentences without causality, they must be missing either a …

Content truncated to 500 words