Fine-Grained Analysis of Cross-Linguistic Syntactic Divergences

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 97–106 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 97 Slot-consistent NLG for Task-oriented Dialogue Systems with Iterative Rectification Network Yangming Li1,2∗, Kaisheng Yao1∗, Libo Qin2∗, Wangxiang Che2†, Xiaolong Li1, Ting Liu2 1Ant Financial Services Group, Alibaba Group 2Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology {pangmao.lym,kaisheng.yao,xl.li}@antfin.com {lbqin,car,tliu}@ir.hit.edu.cn Abstract Data-driven approaches using neural networks have achieved promising performances in natu- ral language generation (NLG). However, neu- ral generators are prone to make mistakes, e.g., neglecting an input slot value and generating a redundant slot value. Prior works refer this to hallucination phenomenon. In this paper, we study slot consistency for building reliable NLG systems with all slot values of input di- alogue act (DA) properly generated in output sentences. We propose Iterative Rectification Network (IRN) for improving general NLG systems to produce both correct and fluent re- sponses. It applies a bootstrapping algorithm to sample training candidates and uses rein- forcement learning to incorporate discrete re- ward related to slot inconsistency into training. Comprehensive studies have been conducted on multiple benchmark datasets, showing that the proposed methods have significantly re- duced the slot error rate (ERR) for all strong baselines. Human evaluations also have con- firmed its effectiveness. 1 Introduction Natural Language Generation (NLG), as a criti- cal component of task-oriented dialogue systems, converts a meaning representation, i.e., dialogue act (DA), into natural language sentences. Tra- ditional methods (Stent et al., 2004; Konstas and Lapata, 2013; Wong and Mooney, 2007) are mostly pipeline-based, dividing the generation process into sentence planing and surface realization. Despite their robustness, they heavily rely on handcrafted rules and domain-specific knowledge. In addition, the generated sentences of rule-based approaches are rather rigid, without the variance of human lan- guage. More recently, neural network based mod- els (Wen et al., 2015a,b; Duˇsek and Jurˇc´ıˇcek, 2016; ∗Equal contributions. † Corresponding author. Input DA inform(NAME = pickwick hotel, PRICERANGE = moderate) Reference the hotel named pickwick hotel is in a moderate price range Missing this is a moderate hotel [NAME] Misplace the pickwick hotel in fort mason is a moderate price range [AREA] Table 1: An exmaple (including mistaken generations) extracted from SF Hotel (Wen et al., 2015b) dataset. Er- rors are marked in colors (missing, misplaced). Tran and Nguyen, 2017a) have attracted much at- tention. They implicitly learn sentence planning and surface realisation end-to-end with cross en- tropy objectives. For example, Duˇsek and Jurˇc´ıˇcek (2016) employ an attentive encoder-decoder model, which applies attention mechanism over input slot value pairs. Although neural generators can be trained end-to-end, they suffer from hallucination phe- nomenon (Balakrishnan et al., 2019). Examples in Table 1 show a misplacement error of an un- seen slot AREA and a missing error of slot NAME by an end-to-end trained model, when compared against its input DA. Motivated by this observa- tion, in this paper, we define slot consistency of NLG systems as all slot values of input …

Content truncated to 500 words