Interpreting Neural Networks to Improve Politeness Comprehension

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2035–2041, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Interpreting Neural Networks to Improve Politeness Comprehension Malika Aubakirova University of Chicago aubakirova@uchicago.edu Mohit Bansal UNC Chapel Hill mbansal@cs.unc.edu Abstract We present an interpretable neural network ap- proach to predicting and understanding polite- ness in natural language requests. Our mod- els are based on simple convolutional neural networks directly on raw text, avoiding any manual identification of complex sentiment or syntactic features, while performing bet- ter than such feature-based models from pre- vious work. More importantly, we use the challenging task of politeness prediction as a testbed to next present a much-needed under- standing of what these successful networks are actually learning. For this, we present sev- eral network visualizations based on activa- tion clusters, first derivative saliency, and em- bedding space transformations, helping us au- tomatically identify several subtle linguistics markers of politeness theories. Further, this analysis reveals multiple novel, high-scoring politeness strategies which, when added back as new features, reduce the accuracy gap be- tween the original featurized system and the neural model, thus providing a clear quantita- tive interpretation of the success of these neu- ral networks. 1 Introduction Politeness theories (Brown and Levinson, 1987; Gu, 1990; Bargiela-Chiappini, 2003) include key com- ponents such as modality, indirection, deference, and impersonalization. Positive politeness strate- gies focus on making the hearer feel good through offers, promises, and jokes. Negative politeness examples include favor seeking, orders, and re- quests. Differentiating among politeness types is a highly nontrivial task, because it depends on fac- tors such as a context, relative power, and culture. Danescu-Niculescu-Mizil et al. (2013) proposed a useful computational framework for predicting po- liteness in natural language requests by designing various lexical and syntactic features about key po- liteness theories, e.g., first or second person start vs. plural. However, manually identifying such polite- ness features is very challenging, because there ex- ist several complex theories and politeness in natu- ral language is often realized via subtle markers and non-literal cues. Neural networks have been achieving high perfor- mance in sentiment analysis tasks, via their ability to automatically learn short and long range spatial relations. However, it is hard to interpret and ex- plain what they have learned. In this paper, we first propose to address politeness prediction via sim- ple CNNs working directly on the raw text. This helps us avoid the need for any complex, manually- defined linguistic features, while still performing better than such featurized systems. More impor- tantly, we next present an intuitive interpretation of what these successful neural networks are learning, using the challenging politeness task as a testbed. To this end, we present several visualization strategies: activation clustering, first derivative saliency, and embedding space transformations, some of which are inspired by similar strategies in computer vision (Erhan et al., 2009; Simonyan et al., 2014; Girshick et al., 2014), and have also been recently adopted in NLP for recurrent neural net- …

Content truncated to 500 words