Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5332–5344 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5332 Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates Katherine A. Keith, David Jensen, and Brendan O’Connor College of Information and Computer Sciences University of Massachusetts Amherst {kkeith,jensen,brenocon}@cs.umass.edu Abstract Many applications of computational social sci- ence aim to infer causal conclusions from non- experimental data. Such observational data often contains confounders, variables that in- fluence both potential causes and potential ef- fects. Unmeasured or latent confounders can bias causal estimates, and this has motivated interest in measuring potential confounders from observed text. For example, an individ- ual’s entire history of social media posts or the content of a news article could provide a rich measurement of multiple confounders. Yet, methods and applications for this prob- lem are scattered across different communi- ties and evaluation practices are inconsistent. This review is the first to gather and categorize these examples and provide a guide to data- processing and evaluation decisions. Despite increased attention on adjusting for confound- ing using text, there are still many open prob- lems, which we highlight in this paper. 1 Introduction In contrast to descriptive or predictive tasks, causal inference aims to understand how intervening on one variable affects another variable (Holland, 1986; Pearl, 2000; Morgan and Winship, 2015; Im- bens and Rubin, 2015; Hern´an and Robins, 2020). Specifically, many applied researchers aim to esti- mate the size of a specific causal effect, the effect of a single treatment variable on an outcome variable. However, a major challenge in causal inference is addressing confounders, variables that influence both treatment and outcome. For example, consider estimating the size of the causal effect of smoking (treatment) on life expectancy (outcome). Occupa- tion is a potential confounder that may influence both the propensity to smoke and life expectancy. Estimating the effect of treatment on outcome with- out accounting for this confounding could result in Figure 1: Left: A causal diagram for text that encodes causal confounders, the setting that is focus of this re- view paper. The major assumption is that latent con- founders can be measured from text and those con- founder measurements can be used in causal adjust- ments. Right: An example application in which practi- tioner does not have access to the confounding variable, occupation, in structured form but can measure con- founders from unstructured text (e.g. an individual’s so- cial media posts). strongly biased estimates and thus invalid causal conclusions. To eliminate confounding bias, one approach is to perform randomized controlled trials (RCTs) in which researchers randomly assign treatment. Yet, in many research areas such as healthcare, educa- tion, or economics, randomly assigning treatment is either infeasible or unethical. For instance, in our running example, one cannot ethically randomly assign participants to smoke since this could ex- pose them to major health risks. In such cases, re- searchers instead use observational data and adjust for the …

Content truncated to 500 words