Generating Long and Informative Reviews with Aspect-Aware Coarse-to-Fine Decoding

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1969–1979 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1969 Generating Long and Informative Reviews with Aspect-Aware Coarse-to-Fine Decoding Junyi Li1, Wayne Xin Zhao1,2∗, Ji-Rong Wen1,2, and Yang Song3 1School of Information, Renmin University of China 2Beijing Key Laboratory of Big Data Management and Analysis Methods 3Boss Zhipin {lijunyi,jrwen}@ruc.edu.cn batmanfly@gmail.com songyang@kanzhun.com Abstract Generating long and informative review text is a challenging natural language generation task. Previous work focuses on word-level generation, neglecting the importance of top- ical and syntactic characteristics from natural languages. In this paper, we propose a novel review generation model by characterizing an elaborately designed aspect-aware coarse-to- fine generation process. First, we model the aspect transitions to capture the overall content flow. Then, to generate a sentence, an aspect- aware sketch will be predicted using an aspect- aware decoder. Finally, another decoder fills in the semantic slots by generating correspond- ing words. Our approach is able to jointly utilize aspect semantics, syntactic sketch, and context information. Extensive experiments results have demonstrated the effectiveness of the proposed model. 1 Introduction In the past decades, online review services (e.g., AMAZON and YELP) have been an important kind of information platforms where users post their feedbacks or comments about products (Kim et al., 2016). Usually, writing an informative and well- structured review will require considerable efforts by users. To assist the writing process, the task of review generation has been proposed to auto- matically generate review text for a user given a product and her/his rating on it (Tang et al., 2016; Zhou et al., 2017). In the literature, various methods have been developed for review generation (Tang et al., 2016; Zhou et al., 2017; Ni et al., 2017; Wang and Zhang, 2017; Catherine and Cohen, 2018). Most of these methods adopt Recurrent Neu- ral Networks (RNN) based methods, especially ∗Corresponding author the improved variants of Long-Short Term Mem- ory (LSTM) (Hochreiter and Schmidhuber, 1997) and Gated Recurrent Unit (GRU) (Cho et al., 2014). They fulfill the review generation task by performing the decoding conditioned on useful context information. Usually, an informative re- view is likely to consist of multiple sentences, con- taining substantive comments from users. Hence, a major problem of existing RNN-based methods is that they have limited capacities in producing long and informative text. More recently, Genera- tive Adversarial Net (GAN) based methods (Zang and Wan, 2017; Yu et al., 2017; Guo et al., 2018; Xu et al., 2018a) have been proposed to enhance the generation of long, diverse and novel text. However, they still focus on word-level genera- tion, and neglect the importance of topical and syntactic characteristics from natural languages. As found in the literature of linguistics (Pullum, 2010) and writing (Bateman and Zock, 2003), the writing process itself has involved multiple stages focusing on different levels of goals. We argue that an ideal review generation approach should follow the writing procedure of a real user and …

Content truncated to 500 words