Pretraining the Noisy Channel Model for Task-Oriented Dialogue

Pretraining the Noisy Channel Model for Task-Oriented Dialogue Qi Liu2∗, Lei Yu1, Laura Rimell1, and Phil Blunsom1,2 1DeepMind, United Kingdom 2University of Oxford, United Kingdom qi.liu@cs.ox.ac.uk {leiyu,laurarimell,pblunsom}@google.com Abstract Direct decoding for task-oriented dialogue is known to suffer from the explaining-away ef- fect, manifested in models that prefer short and generic responses. Here we argue for the use of Bayes’ theorem to factorize the dialogue task into two models, the distribution of the context given the response, and the prior for the response itself. This approach, an instan- tiation of the noisy channel model, both miti- gates the explaining-away effect and allows the principled incorporation of large pretrained models for the response prior. We present ex- tensive experiments showing that a noisy chan- nel model decodes better responses compared to direct decoding and that a two-stage pre- training strategy, employing both open-domain and task-oriented dialogue data, improves over randomly initialized models. 1 Introduction Task-oriented dialogue agents provide a conver- sational interface to assist users in accomplish- ing specific goals, such as finding a restaurant or booking a hotel (Seneff and Polifroni, 2000; Raux et al., 2005; Budzianowski et al., 2018; Peng et al., 2020a). Increasing demand from indus- try for natural language assistants and scalable cus- tomer service solutions has recently been driving a renaissance in the development of task-oriented dialogue models. In addition, the specification of explicit dialogue agent goals, afforded by the task-oriented paradigm, makes such research eas- ier to ground and evaluate than open-domain chatbots. Current research on task-oriented dialogue is dominated by monolithic sequence-to-sequence models that directly parameterize the conditional distribution of the response given the prior dia- ∗Work completed during an internship at DeepMind. logue context. However, this monolithic approach conflates the task-specific and language-general aspects of dialogue, and adversely favors short and generic responses (Bao et al., 2020) due to the explaining-away effect (Klein and Manning, 2002). Here we pursue an alternative to the direct model. Using Bayes’ rule allows us to factorize the probability of the response given the context p(R| C) into a language model p(R) and a context model p(C| R).1 Within natural language processing (NLP), this approach is traditionally known as the noisy channel model (Shannon, 1948), and has recently seen renewed interest with its successful application to neural machine translation (Yu et al., 2017, 2020; Yee et al., 2019). We hypothesize that the noisy channel reformu- lation is advantageous for dialogue because the factorization enables each sub-module to special- ize in a dialogue sub-task. In particular, the con- text conditional model can help to discount short and generic responses and mitigate the explaining- away effect, while the language model helps ensure that responses are natural. We find that a noisy channel model with the same number of parameters as a direct model achieves better ac- curacy on three task-oriented dialogue datasets. Moreover, a larger noisy channel model can be trained with the same hardware, by training the sub-modules separately, yielding additional improvements. It has become common in recent years to …

Content truncated to 500 words