Injecting Numerical Reasoning Skills into Language Models
38278Authors
nan
Abstract
AbstractLarge pre-trained language models (LMs) are known to encode substantial amounts of linguistic information. However, high-level reasoning skills, such as numerical reasoning, are difficult to learn from a language-modeling objective only. Consequently, existing models for numerical reasoning have used specialized architectures with limited flexibility. In this work, we show that numerical reasoning is amenable to automatic data generation, and thus one can inject this skill into pre-trained LMs, by generating large amounts of data, and training in a multi-task setup. We show that pre-training our model, GenBERT, on this data, dramatically improves performance on DROP (49.3 –> 72.3 F1), reaching performance that matches state-of-the-art models of comparable size, while using a simple and general-purpose encoder-decoder architecture. Moreover, GenBERT generalizes well to math word problem datasets, while maintaining high performance on standard RC tasks. Our approach provides a general recipe for injecting skills into large pre-trained LMs, whenever the skill is amenable to automatic data augmentation.
Keywords
Full Content
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 66–84 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 66 Large Scale Multi-Actor Generative Dialog Modeling Alex Boyd∗ † Department of Statistics University of California, Irvine alexjb@uci.edu Raul Puri∗ NVIDIA raulp@nvidia.com Mohammad Shoeybi NVIDIA mshoeybi@nvidia.com Mostofa Patwary NVIDIA mpatwary@nvidia.com Bryan Catanzaro NVIDIA bcatanzaro@nvidia.com Abstract Non-goal oriented dialog agents (i.e. chat- bots) aim to produce varying and engaging conversations with a user; however, they typ- ically exhibit either inconsistent personality across conversations or the average personal- ity of all users. This paper addresses these issues by controlling an agent’s persona upon generation via conditioning on prior conversa- tions of a target actor. In doing so, we are able to utilize more abstract patterns within a person’s speech and better emulate them in generated responses. This work introduces the GENERATIVE CONVERSATION CONTROL model, an augmented and fine-tuned GPT-2 language model that conditions on past refer- ence conversations to probabilistically model multi-turn conversations in the actor’s per- sona. We introduce an accompanying data collection procedure to obtain 10.3M conver- sations from 6 months worth of Reddit com- ments. We demonstrate that scaling model sizes from 117M to 8.3B parameters yields an improvement from 23.14 to 13.14 perplexity on 1.7M held out Reddit conversations. In- creasing model scale yielded similar improve- ments in human evaluations that measure pref- erence of model samples to the held out target distribution in terms of realism (31% increased to 37% preference), style matching (37% to 42%), grammar and content quality (29% to 42%), and conversation coherency (32% to 40%). We find that conditionally modeling past conversations improves perplexity by 0.47 in automatic evaluations. Through human tri- als we identify positive trends between condi- tional modeling and style matching and outline steps to further improve persona control. ∗First two authors have contributed equally. †Research conducted during an internship at NVIDIA. 1 Introduction Modeling dialog agents, otherwise known as chat- bots, has been a longstanding goal within artificial intelligence research. Historically, approaches to this task can be divided into one of the two cat- egories: retrieval and generative. The former is posed as a search problem where an appropriate response for a conversation is selected from a large set of candidate replies, whereas the latter autore- gressively samples a reply, thereby potentially cre- ating a response that the model may not have seen before. The flexibility and creativity afforded by not prespecifying every possible response is a ma- jor draw for generative based approaches. In recent years, advances in neural methods have shown promise in effectively modeling this task. Early progress first demonstrated potential with recurrent network based models capable of hold- ing simple conversations (Sordoni et al., 2015). Further architecture optimizations and tweaks im- proved user experiences; however, they largely ex- perienced issues with the agent exhibiting an in- consistent personality and producing uninteresting comments (Li et al., 2015). Some works have at- tempted to alleviate this through conditioning on various factors of the …
Content truncated to 500 words
Labeled Text
AbstractLarge pre-trained language models (LMs) are known to encode substantial amounts of linguistic information. However, high-level reasoning skills, such as numerical reasoning, are difficult to learn from a language-modeling objective only. Consequently, existing models for numerical reasoning have used specialized architectures with limited flexibility. In this work, we show that numerical reasoning is amenable to automatic data generation, and thus one can inject this skill into pre-trained LMs, by generating large amounts of data, and training in a multi-task setup. We show that pre-training our model, GenBERT, on this data, dramatically improves performance on DROP (49.3 –> 72.3 F1), reaching performance that matches state-of-the-art models of comparable size, while using a simple and general-purpose encoder-decoder architecture. Moreover, GenBERT generalizes well to math word problem datasets, while maintaining high performance on standard RC tasks. Our approach provides a general recipe for injecting skills into large pre-trained LMs, whenever the skill is amenable to automatic data augmentation.
Publication Details
- Paper ID:
- 38278
- Year:
- 2020
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan