Semantic Role Labeling Guided Multi-turn Dialogue ReWriter

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6632–6639, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6632 Semantic Role Labeling Guided Multi-turn Dialogue ReWriter Kun Xu1, Haochen Tan2, Linfeng Song1, Han Wu2, Haisong Zhang1, Linqi Song2, Dong Yu1 1Tencent AI Lab 2City University of Hong Kong {kxkunxu,lfsong,hansongzhang,dyu}@tencent.com {haochetan2-c,hanwu32-c}@my.cityu.edu.hk {linqi.song}@cityu.edu.hk Abstract For multi-turn dialogue rewriting, the capacity of effectively modeling the linguistic knowl- edge in dialog context and getting rid of the noises is essential to improve its performance. Existing attentive models attend to all words without prior focus, which results in inaccu- rate concentration on some dispensable words. In this paper, we propose to use semantic role labeling (SRL), which highlights the core se- mantic information of who did what to whom, to provide additional guidance for the rewriter model. Experiments show that this informa- tion significantly improves a RoBERTa-based model that already outperforms previous state- of-the-art systems. 1 Introduction Recent research (Vinyals and Le, 2015; Li et al., 2016; Serban et al., 2017; Zhao et al., 2017; Shao et al., 2017) on dialogue generation has been achieving impressive progress for making single- turn responses, while producing coherent multi- turn replies still remains extremely challenging. One important factor that contributes to this dif- ficulty is coreference and information omission, where mention is dropped or replaced by a pro- noun for simplicity. These phenomena dramati- cally introduce the requirements for long-distance reasoning, as they frequently occurred in our daily conversations, especially in pro-drop languages like Chinese and Japanese. To tackle these problems, sentence rewriting was introduced to ease the burden of dialogue models by simplifying the multi-turn dialogue modeling into a single-turn problem. Several ap- proaches (Su et al., 2019; Zhang et al., 2019; El- gohary et al., 2019) have been proposed to address the rewriting task. Conceptually, these models fol- low the conventional encoder-decoder architecture Utterance 1 需要粤语 (I may need Cantonese.) Utterance 2 粤语ARG0是普通话ARG1吗 (Is Cantonese Mandarin ?) Utterance 3 不算predicate吧 (Maybe Not.) Utterance 3′ 粤 粤 粤语 语 语不 不 不算 算 算普 普 普通 通 通话 话 话吧 吧 吧 (Cantonese may be not Mandarin.) Table 1: One example of multi-turn dialogue. The goal of dialogue rewriting is to rewrite utterance 3 into 3′. that first encodes the dialogue context into a dis- tributional representation and then decodes it to the rewritten utterance. Their decoders mainly use global attention methods that attends to all words in the dialogue context without prior focus, which may result in inaccurate concentration on some dispensable words. We also observe that the accu- racy of their models significantly decreases when working on long dialogue contexts. This observa- tion is expected since if the text is lengthy, it would be quite difficult for deep learning models to un- derstand as it suffers from noise and pays vague attention to the text components. Motivated by these observations, we propose to incorporate the information of Semantic role la- beling (SRL) (Gildea and Jurafsky, 2002; Palmer et al., 2010) to …

Content truncated to 500 words