Deconfounded Lexicon Induction for Interpretable Social Science

Proceedings of NAACL-HLT 2018, pages 1615–1625 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Deconfounded Lexicon Induction for Interpretable Social Science Reid Pryzant∗, Kelly Shen∗, Dan Jurafsky+†, Stefan Wager‡ ∗+Department of Computer Science †Department of Linguistics ‡Graduate School of Business Stanford University {rpryzant,kshen21,jurafsky,swager}@stanford.edu Abstract NLP algorithms are increasingly used in com- putational social science to take linguistic ob- servations and predict outcomes like human preferences or actions. Making these social models transparent and interpretable often re- quires identifying features in the input that pre- dict outcomes while also controlling for po- tential confounds. We formalize this need as a new task: inducing a lexicon that is predic- tive of a set of target variables yet uncorre- lated to a set of confounding variables. We introduce two deep learning algorithms for the task. The first uses a bifurcated architecture to separate the explanatory power of the text and confounds. The second uses an adversarial discriminator to force confound-invariant text encodings. Both elicit lexicons from learned weights and attentional scores. We use them to induce lexicons that are predictive of timely responses to consumer complaints (controlling for product), enrollment from course descrip- tions (controlling for subject), and sales from product descriptions (controlling for seller). In each domain our algorithms pick words that are associated with narrative persuasion; more predictive and less confound-related than those of standard feature weighting and lexi- con induction techniques like regression and log odds. 1 Introduction Applications of NLP to computational social sci- ence and data science increasingly use lexical fea- tures (words, prefixes, etc) to help predict non- linguistic outcomes like sales, stock prices, hospi- tal readmissions, and other human actions or pref- erences. Lexical features are useful beyond pre- dictive performance. They enhance interpretabil- ity in machine learning because practitioners know why their system works. Lexical features can also be used to understand the subjective properties of a text. For social models, we need to be able to select lexical features that predict the desired outcome(s) while also controlling for potential confounders. For example, we might want to know which words in a product description lead to greater sales, re- gardless of the item’s price. Words in a description like “luxury” or “bargain” might increase sales but also interact with our confound (price). Such words don’t reflect the unique part of text’s ef- fect on sales and should not be selected. Simi- larly, we might want to know which words in a consumer complaint lead to speedy administrative action, regardless of the product being complained about; which words in a course description lead to higher student enrollment, regardless of the course topic. These instances are associated with narra- tive persuasion: language that is responsible for altering cognitive responses or attitudes (Spence, 1983; Van Laer et al., 2013). In general, we want words which are predictive of their targets yet decorrelated from confound- ing information. The lexicons constituted by these words are useful in their own right (to develop causal domain theories …

Content truncated to 500 words