Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 831–844 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 831 Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue Anusha Balakrishnan∗Jinfeng Rao* Kartikeya Upasani* Michael White*† and Rajen Subba* Facebook Conversational AI {anushabala,raojinfeng,kart,mwhite14850,rasubba}@fb.com Abstract Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems. Avenues like the E2E NLG Chal- lenge have encouraged the development of neural approaches, particularly sequence-to- sequence (Seq2Seq) models for this problem. The semantic representations used, however, are often underspecified, which places a higher burden on the generation model for sentence planning, and also limits the extent to which generated responses can be controlled in a live system. In this paper, we (1) propose using tree-structured semantic representations, like those used in traditional rule-based NLG systems, for better discourse-level structuring and sentence-level planning; (2) introduce a challenging dataset using this representation for the weather domain; (3) introduce a con- strained decoding approach for Seq2Seq mod- els that leverages this representation to im- prove semantic correctness; and (4) demon- strate promising results on our dataset and the E2E dataset. 1 Introduction Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems. With their end-to-end trainability, neural approaches to natural language generation (NNLG), partic- ularly sequence-to-sequence (Seq2Seq) models, have been promoted with great fanfare in recent years (Wen et al., 2015, 2016; Mei et al., 2016; Kiddon et al., 2016; Duˇsek and Jurcicek, 2016), and avenues like the recent E2E NLG challenge (Duˇsek et al., 2018, 2019) have made available large datasets to promote the development of these models. Nevertheless, current NNLG models arguably remain inadequate for most real-world ∗Alphabetical by first name †Work done while on leave from Ohio State University task-oriented dialogue systems, given their inabil- ity to (i) reliably perform common sentence plan- ning and discourse structuring operations (Reed et al., 2018), (ii) generalize to complex inputs (Wiseman et al., 2017), and (3) avoid generating texts with semantic errors including hallucinated content (Duˇsek et al., 2018, 2019).1 In this paper, we explore the extent to which these issues can be addressed by incorporating lessons from pre-neural NLG systems into a neu- ral framework. We begin by arguing in favor of enriching the input to neural generators to in- clude discourse relations — long taken to be cen- tral in traditional NLG — and underscore the im- portance of exerting control over these relations when generating text, particularly when using user models to structure responses. In a closely re- lated work, Reed et al. (2018), the authors add control tokens (to indicate contrast and sentence structure) to a flat input MR, and show that these can be effectively used to control structure. How- ever, their methods are only able to control the presence or absence of these relations, without more fine-grained control over their structure. We thus go beyond their approach and …

Content truncated to 500 words