Extractive Opinion Summarization in Quantized Transformer Spaces

Extractive Opinion Summarization in Quantized Transformer Spaces Stefanos Angelidis1 Reinald Kim Amplayo1 Yoshihiko Suhara2 Xiaolan Wang2 Mirella Lapata1 1University of Edinburgh 2Megagon Labs s.angelidis@ed.ac.uk, reinald.kim@ed.ac.uk yoshi@megagon.ai, xiaolan@megagon.ai mlap@inf.ed.ac.uk Abstract We present the Quantized Transformer (QT), an unsupervised system for extractive opi- nion summarization. QT is inspired by Vector- Quantized Variational Autoencoders, which we repurpose for popularity-driven summari- zation. It uses a clustering interpretation of the quantized space and a novel extraction algo- rithm to discover popular opinions among hun- dreds of reviews, a significant step towards opinion summarization of practical scope. In addition, QT enables controllable summari- zation without further training, by utilizing properties of the quantized space to extract aspect-specific summaries. We also make pub- licly available SPACE, a large-scale evaluation benchmark for opinion summarizers, com- prising general and aspect-specific summaries for 50 hotels. Experiments demonstrate the promise of our approach, which is validated by human studies where judges showed clear preference for our method over competitive baselines. 1 Introduction Online reviews play an integral role in modern life, as we look to previous customer experiences to inform everyday decisions. The need to digest review content has fueled progress in opinion mining (Pang and Lee, 2008), whose central goal is to automatically summarize people’s attitudes towards an entity. Early work (Hu and Liu, 2004) focused on numerically aggregating customer satisfaction across different aspects of the entity under consideration (e.g., the quality of a camera, its size, clarity). More recently, the success of neural summarizers in the Wikipedia and news domains (Cheng and Lapata, 2016; See et al., 2017; Narayan et al., 2018; Liu et al., 2018; Perez- Beltrachini etal., 2019) has spurred interest in opinion summarization; the aggregation, in textual form, of opinions expressed in a set of reviews (Angelidis and Lapata, 2018; Huy Tien et al., 2019; Tian et al., 2019; Coavoux et al., 2019; Chu and Liu, 2019; Isonuma et al., 2019; Braˇzinskas et al., 2020; Amplayo and Lapata, 2020; Suhara et al., 2020; Wang et al., 2020). Opinion summarization has distinct character- istics that set it apart from other summarization tasks. First, it cannot rely on reference summaries for training, because such meta-reviews are very scarce and their crowdsourcingis unfeasible. Even for a single entity, annotators would have to pro- duce summaries after reading hundreds, some- times thousands, of reviews. Second, the inherent subjectivity of review text distorts the notion of information importance used in generic summa- rization (Peyrard, 2019). Conflicting opinions are often expressed for the same entity and, therefore, useful summaries should be based on opinion pop- ularity (Ganesan et al., 2010). Moreover, methods need to be flexible with respect to the size of the input (entities are frequently reviewed by thou- sands of users), and controllable with respect to the scope of the output. For instance, users may wish to read a general overview summary, or a more targeted one about a particular aspect of interest (e.g., a hotel’s location, its cleanliness, or available food options). Recent work (Tian et al., 2019; Coavoux …

Content truncated to 500 words