Low-Dimensional Hyperbolic Knowledge Graph Embeddings

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 650–659 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 650 Response-Anticipated Memory for On-Demand Knowledge Integration in Response Generation Zhiliang Tian,∗ 1,4 Wei Bi,† 2 Dongkyu Lee,1 Lanqing Xue,1 Yiping Song,3 Xiaojiang Liu,2 Nevin L. Zhang1,4 1Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, China 2Tencent AI Lab, Shenzhen, China 3Department of Computer Science School of EECS, Peking University, Beijing, China 4HKUST Xiao-i Robot Joint Lab, Hong Kong SAR, China {ztianac,dleear,lxueaa,lzhang}@cse.ust.hk {victoriabi,kieranliu}@tencent.com songyiping@pku.edu.cn Abstract Neural conversation models are known to generate appropriate but non-informative re- sponses in general. A scenario where infor- mativeness can be significantly enhanced is Conversing by Reading (CbR), where conver- sations take place with respect to a given exter- nal document. In previous work, the external document is utilized by (1) creating a context- aware document memory that integrates in- formation from the document and the conver- sational context, and then (2) generating re- sponses referring to the memory. In this paper, we propose to create the document memory with some anticipated responses in mind. This is achieved using a teacher-student framework. The teacher is given the external document, the context, and the ground-truth response, and learns how to build a response-aware doc- ument memory from three sources of infor- mation. The student learns to construct a response-anticipated document memory from the first two sources, and the teacher’s insight on memory creation. Empirical results show that our model outperforms the previous state- of-the-art for the CbR task. 1 Introduction Neural conversation models have achieved promis- ing performance in response generation. However, it is widely observed that the generated responses lack sufficient content and information (Li et al., 2016a). One way to address this issue is to in- tegrate various external information into conver- sation models. Examples of external information include document topics (Xing et al., 2017), com- monsense knowledge graphs (Zhou et al., 2018), and domain-specific knowledge bases (Yang et al., 2019). Conversing by reading (CbR) (Qin et al., ∗This work was partially done when Zhiliang Tian was an intern at Tencent AI Lab. †Corresponding author But, he’s struggling with diseases now. Context Memory Jackie was a renowned actor and starred many films, so he had many fans. He’s generous and wealthy. Document Jackie can afford best treatment. Fans will pray for him. Response … Jackie was a wealthy Figure 1: A motivating example of constructing a response-anticipated document memory for response generation. Details are provided in the introduction. 2019) is a recently proposed scenario where exter- nal information can be ingested to conversations. In CbR, conversations take place with reference to a document. The key problem in CbR is to learn how to integrate information from the external doc- ument into response generation on demand. To exploit knowledge from documents for con- versations, a conventional way is to extend the sequence-to-sequence (Seq2Seq) model (Sutskever et al., 2014) with Memory Networks (Sukhbaatar …

Content truncated to 500 words