Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 115–125 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 115 Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generation Chujie Zheng†, Yunbo Cao‡, Daxin Jiang§, Minlie Huang†∗ †Department of Computer Science and Technology, †Institute for Artificial Intelligence, †State Key Lab of Intelligent Technology and Systems, †Beijing National Research Center for Information Science and Technology, †Tsinghua University, Beijing 100084, China ‡Smart Platform Product Department, Tencent, Beijing §STCA NLP Group, Microsoft, Beijing chujiezhengchn@gmail.com, yunbocao@tencent.com djiang@microsoft.com, aihuang@tsinghua.edu.cn Abstract In a multi-turn knowledge-grounded dialog, the difference between the knowledge selected at different turns usually provides potential clues to knowledge selection, which has been largely neglected in previous research. In this paper, we propose a difference-aware knowl- edge selection method. It first computes the difference between the candidate knowledge sentences provided at the current turn and those chosen in the previous turns. Then, the differential information is fused with or disentangled from the contextual information to facilitate final knowledge selection. Au- tomatic, human observational, and interactive evaluation shows that our method is able to select knowledge more accurately and gener- ate more informative responses, significantly outperforming the state-of-the-art baselines. The codes are available at https://github. com/chujiezheng/DiffKS. 1 Introduction Knowledge-grounded conversation generation aims at generating informative responses based on both discourse context and external knowledge (Ghazvininejad et al., 2018; Zhou et al., 2018a), where selecting appropriate knowledge is critical to the success of the task. Existing knowledge se- lection models generally fall into two types. One type is solely based on the context (Lian et al., 2019; Zhang et al., 2019; Meng et al., 2020; Ren et al., 2020), which we call non-sequential selec- tion because knowledge selection at different turns is independent. The other type sequentially selects knowledge additionally conditioned on previously selected knowledge (Kim et al., 2020), which we ∗* Corresponding author. Do you like to read? Yes, I do like to read. It is the process of decoding symbols in order to derive meaning. Reading is a complex "cognitive process" of decoding symbols in order to construct or derive meaning. 1. Reading is a complex "cognitive process" of decoding symbols in order to construct or derive meaning. That's right. I was thinking of picking up the second Mistborn book and finishing the series. I haven't read Mistborn myself but heard they are fantasy novels written by Brandon Sanderson. 3. List decoding is an alternative to unique decoding of error-correcting codes for large error rates. 2. Mistborn is a series of epic fantasy novels written by American author Brandon Sanderson. Figure 1: An example of difference-aware knowledge selection. The blue △ denotes that the corresponding knowledge has little difference from or is identical to the previously selected one, and selecting it may lead to repetitive responses. The red × denotes that the dif- ference is too large, and selecting it could make the response incoherent with the context. call sequential selection. As shown in Kim et al. (2020), such a sequential way can better …

Content truncated to 500 words