Explain Yourself! Leveraging Language Models for Commonsense Reasoning

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4932–4942 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4932 Explain Yourself! Leveraging Language Models for Commonsense Reasoning Nazneen Fatema Rajani Bryan McCann Caiming Xiong Richard Socher Salesforce Research Palo Alto, CA, 94301 {nazneen.rajani,bmccann,cxiong,rsocher}@salesforce.com Abstract Deep learning models perform poorly on tasks that require commonsense reasoning, which often necessitates some form of world- knowledge or reasoning over information not immediately present in the input. We collect human explanations for commonsense reason- ing in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations (CoS-E). We use CoS-E to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Expla- nation (CAGE) framework. CAGE improves the state-of-the-art by 10% on the challeng- ing CommonsenseQA task. We further study commonsense reasoning in DNNs using both human and auto-generated explanations in- cluding transfer to out-of-domain tasks. Em- pirical results indicate that we can effectively leverage language models for commonsense reasoning. 1 Introduction Commonsense reasoning is a challenging task for modern machine learning methods (Zhong et al., 2018; Talmor et al., 2019). Explanations are a way to verbalize the reasoning that the models learn during training. Common sense Question Answer- ing (CQA) is a multiple-choice question answer- ing dataset proposed for developing natural lan- guage processing (NLP) models with commons- sense reasoning capabilities (Talmor et al., 2019). Although these efforts have led to progress, it is still unclear how these models perform reasoning and to what extent that reasoning is based on world knowledge. We collect human explanations for commonsense reasoning built on top of CQA and introduce them as Common Sense Explanations (CoS-E)1. CoS-E contains human explanations in 1https://github.com/nazneenrajani/CoS-E Question: While eating a hamburger with friends, what are people trying to do? Choices: have fun, tasty, or indigestion CoS-E: Usually a hamburger with friends indicates a good time. Question: After getting drunk people couldn’t understand him,it was because of his what? Choices: lower standards,slurred speech, or falling down CoS-E: People who are drunk have difficulty speaking. Question: People do what during their time off from work? Choices: take trips, brow shorter, or become hysterical CoS-E: People usually do something relaxing, such as taking trips,when they don’t need to work. Table 1: Examples from our CoS-E dataset. the form of both open-ended natural language ex- planations as well as highlighted span annotations that represent words selected by humans as impor- tant for predicting the right answer (see Table 1). Talmor et al. (2019) show that using Google search to extract context from top 100 result snip- pets for each of the question and answer choices does not help much in improving the accuracy on CQA trained using even the state-of-the-art read- ing comprehension model BiDAF++ (Seo et al., 2017) augmented with a self-attention layer and ELMo representations (Peters et al., 2018). In contrast, we leverage a pretrained language model …

Content truncated to 500 words