GRS: Combining Generation and Revision in Unsupervised Sentence Simplification

Findings of the Association for Computational Linguistics: ACL 2022, pages 68 - 78 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Multi-Scale Distribution Deep Variational Autoencoder for Explanation Generation ZeFeng Cai1, Linlin Wang1∗, Gerard de Melo2, Fei Sun3, Liang He1 1 East China Normal University 2 Hasso Plattner Institute, University of Potsdam 3 Alibaba Group oklen@stu.ecnu.edu.cn, {llwang,lhe}@cs.ecnu.edu.cn, gdm@demelo.org ofey.sunfei@gmail.com Abstract Generating explanations for recommender sys- tems is essential for improving their trans- parency, as users often wish to understand the reason for receiving a specified recommenda- tion. Previous methods mainly focus on im- proving the generation quality, but often pro- duce generic explanations that fail to incor- porate specific details of user and item. To resolve this problem, we present Multi-Scale Distribution Deep Variational Autoencoders (MVAE). A deep hierarchical VAE with a prior network that eliminates noise while retaining meaningful signals in the input, coupled with a recognition network serving as the source of information to guide the learning of the prior network. Further, the Multi-scale dis- tribution Learning Framework (MLF) along with a Target Tracking Kullback-Leibler diver- gence (TKL) mechanism are proposed to em- ploy multiple KL divergences at different scales for more effective learning. Extensive empiri- cal experiments demonstrate that our methods can generate explanations with concrete input- specific contents. 1 Introduction Due to the massive demand for convincing high- quality recommendations, researchers from both academic and industrial communities have paid in- creasing attention to the topic of enhancing the ex- plainability of recommender systems (Wang et al., 2018b,a; Xian et al., 2019; Chen et al., 2019). Ex- planations for recommendations in real-world sce- narios are presented in a variety of different forms, among them, the most popular and natural form is that of free-text explanations given in natural language (Zhang and Chen, 2020). As shown in Fig. 1, this task requires a machine to generate a textual explanation based on a given user ID, item ID, and the rating score from a rec- ommender system. Previous models attempt to ∗ Corresponding author. Email: llwang@cs.ecnu.edu.cn. Figure 1: An example of explanation generation. embed these IDs in a similar way as normal words. However, since the IDs appear far less frequently than the words, most approaches typically fail to account for specific features of the users and item. Hence, it is a very common phenomenon to obtain explanations without concrete characteristics about the given user and item as shown in Table 4. A probable reason for this phenomenon is that these models fail to utilize the input embeddings effec- tively. Specifically, in most models, the user and item information is merely provided as randomly initialized input embeddings, which barely contain meaningful information, but introduce noise that may be indistinguishable from more meaningful information. Here, we refer to noise from the simi- larities of randomly initialized input embeddings that are conflated with implicit patterns contained in our data. For example, there may be two user embedding similar to each other while in our data they represent users very different from …

Content truncated to 500 words