Uncovering Main Causalities for Long-tailed Information Extraction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9683–9695 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9683 Uncovering Main Causalities for Long-tailed Information Extraction Guoshun Nan1∗, Jiaqi Zeng2∗, Rui Qiao3∗, Zhijiang Guo4 and Wei Lu1 1StatNLP Research Group, Singapore University of Technology and Design 2Carnegie Mellon University 3National University of Singapore 4University of Cambridge nanguoshun@gmail.com, jiaqizen@andrew.cmu.edu qiaorui@comp.nus.edu.sg, zg283@cam.ac.uk luwei@sutd.edu.sg Abstract Information Extraction (IE) aims to extract structural information from unstructured texts. In practice, long-tailed distributions caused by the selection bias of a dataset, may lead to in- correct correlations, also known as spurious correlations, between entities and labels in the conventional likelihood models. This moti- vates us to propose counterfactual IE (CFIE), a novel framework that aims to uncover the main causalities behind data in the view of causal inference. Specifically, 1) we first introduce a unified structural causal model (SCM) for various IE tasks, describing the relationships among variables; 2) with our SCM, we then generate counterfactuals based on an explicit language structure to better calculate the direct causal effect during the inference stage; 3) we further propose a novel debiasing approach to yield more robust predictions. Experiments on three IE tasks across five public datasets show the effectiveness of our CFIE model in mitigat- ing the spurious correlation issues. 1 Introduction The goal of Information Extraction (IE) is to de- tect the structured information from unstructured texts. Previous deep learning models for IE tasks, such as named entity recognition (NER; Lample et al. 2016), relation extraction (RE; Peng et al. 2017) and event detection (ED; Nguyen and Gr- ishman 2015), are largely proposed for learning under some reasonably balanced label distributions. However, in practice, these labels usually follow a long-tailed distribution (Doddington et al., 2004). Figure 1 shows such an unbalanced distribution on the ACE2005 (Doddington et al., 2004) dataset. As a result, performance on the instance-scarce (tail) classes may drop significantly. For example, on an existing model for NER (Jie and Lu, 2019), the macro F1 score of instance-rich (head) classes ∗Contributed equally. head classes tail classes 20% training instances 80% training instances Figure 1: Class distribution of the ACE2005 dataset. can be 71.6, while the score of tail classes sharply decreases to 41.7. The underlying causes for the above issue are the biased statistical dependencies between entities1 and classes, known as spurious correlations (Sri- vastava et al., 2020). For example, an entity Gardens appears 13 times in the training set of OntoNotes5.0, with the NER tag location LOC, and only 2 times as organization ORG. A classi- fier trained on this dataset tends to build spurious correlations between Gardens and LOC, although Gardens itself does not indicate a location. Most existing works on addressing spurious correlations focus on images, such as re-balanced training (Lin et al., 2017), transfer learning (Liu et al., 2019) and decoupling (Kang et al., 2019). However, these approaches may not be suitable for natural lan- guage inputs. Recent efforts on information extrac- tion (Han et al., …

Content truncated to 500 words