Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5083–5089, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 5083 Simple Data Augmentation with the MASK Token Improves Domain Adaptation for Dialog Act Tagging Semih Yavuz Kazuma Hashimoto Wenhao Liu Nitish Shirish Keskar Richard Socher Caiming Xiong Salesforce Research {syavuz, k.hashimoto, wenhao.liu, nkeskar, rsocher, cxiong}@salesforce.com Abstract The concept of Dialogue Act (DA) is univer- sal across different task-oriented dialogue do- mains - the act of “request” carries the same speaker intention whether it is for restaurant reservation or flight booking. However, DA taggers trained on one domain do not gener- alize well to other domains, which leaves us with the expensive need for a large amount of annotated data in the target domain. In this work, we investigate how to better adapt DA taggers to desired target domains with only un- labeled data. We propose MASKAUGMENT, a controllable mechanism that augments text in- put by leveraging the pre-trained MASK token from BERT model. Inspired by consistency regularization, we use MASKAUGMENT to in- troduce an unsupervised teacher-student learn- ing scheme to examine the domain adaptation of DA taggers. Our extensive experiments on the Simulated Dialogue (GSim) and Schema- Guided Dialogue (SGD) datasets show that MASKAUGMENT is useful in improving the cross-domain generalization for DA tagging. 1 Introduction Dialog act (DA) tagging, one of the important NLU components of modern task-oriented dialog sys- tems, aims to capture the speaker’s intention behind the utterances at each dialog turn. Several different schema and taxonomies have been introduced by several different researchers (Core and Allen, 1997; Stolcke et al., 2000; Bunt et al., 2010; Mezza et al., 2018) over the years. However, the main focus of the recent work (Kumar et al., 2018; Chen et al., 2018; Raheja and Tetreault, 2019) on DA tagging was on human-human social conversations (God- frey et al., 1992; Jurafsky et al., 1997), which is less applicable for task-oriented setting. Recently, several task-oriented dialogue datasets (Shah et al., 2018; Henderson et al., 2014; Budzianowski et al., 2018) have been released. Figure 1: Overview of dialog act tagging task and cross- domain generalization scenario of similar dialog acts. The specific contents of the utterances of the same dialog act (DA) are distinct due to the domain difference, making the cross- domain generalization challenging. However, the discrepancy in their annotation schema hinders the progress on building DA taggers that can generalize across domains and possibly datasets. To address this issue, Paul et al. (2019) propose a universal schema for DAs by aligning annotations for multiple existing corpora. In this regard, another useful corpora employed as a testbed in this work is Schema-guided dialogues (SGD) (Rastogi et al., 2020), which covers 20 domains under the same DA annotation schema. It is often challenging and costly to obtain a large amount of in-domain dialogues with annotations. However, unlabeled dialogue corpora in target do- main can easily be curated from past conversation logs or collected via crowd-sourcing (Byrne et al., 2019; Budzianowski …

Content truncated to 500 words