Unsupervised Opinion Summarisation in the Wasserstein Space

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8592 - 8607 December 7-11, 2022 ©2022 Association for Computational Linguistics Unsupervised Opinion Summarisation in the Wasserstein Space Jiayu Song1, Iman Munire Bilal2,3, Adam Tsakalidis1,3, Rob Procter2,3, Maria Liakata1,2,3 1 Queen Mary University of London, London, UK 2 University of Warwick, Coventry, UK 3 The Alan Turing Institute, London, UK {jiayu.song,a.tsakalidis,m.liakata}@qmul.ac.uk Abstract Opinion summarisation synthesises opinions expressed in a group of documents discussing the same topic to produce a single summary. Recent work has looked at opinion summari- sation of clusters of social media posts. Such posts are noisy and have unpredictable struc- ture, posing additional challenges for the con- struction of the summary distribution and the preservation of meaning compared to online reviews, which has been so far the focus of opinion summarisation. To address these chal- lenges we present WassOS, an unsupervised abstractive summarization model which makes use of the Wasserstein distance. A Variational Autoencoder is used to get the distribution of documents/posts, and the distributions are dis- entangled into separate semantic and syntactic spaces. The summary distribution is obtained using the Wasserstein barycenter of the seman- tic and syntactic distributions. A latent variable sampled from the summary distribution is fed into a GRU decoder with a transformer layer to produce the final summary. Our experiments on multiple datasets including Twitter clusters, Reddit threads, and reviews show that WassOS almost always outperforms the state-of-the-art on ROUGE metrics and consistently produces the best summaries with respect to meaning preservation according to human evaluations. 1 Introduction The growth of online platforms has encouraged people to share their opinions, such as product re- views on online shopping platforms (e.g., Amazon) and responses to events posted on social media (e.g., Twitter). Summarising users’ opinions over particular topics on such platforms is crucial for decision-making and helping online users find rel- evant information of interest (Rashid et al., 2002; Fan et al., 2019). Specifically multi-document opin- ion summarisation aims at automatically summaris- ing multiple opinions on the same topic (Moussa et al., 2018). The bulk of work in this area uses unsupervised summarisation methods. Datasets/Domains. Most work on unsupervised abstractive opinion summarisation focuses on re- views (e.g., Amazon, Yelp) (Wang and Ling, 2016; Chu and Liu, 2019; Bražinskas et al., 2020; Am- playo and Lapata, 2020; Elsahar et al., 2021). How- ever, it is also important to capture user opinions in online discussions over specific events or topics on popular social media platforms such as Twit- ter (Bilal et al., 2022b) and Reddit, where the text structure and content is very different and often much noisier compared to review-based corpora (see some examples in Appendix A.5 and A.6). Summary Representation. A main focus of unsu- pervised abstractive summarisation is the creation of a meaningful summary representation. Mean- Sum (Chu and Liu, 2019) used a text autoencoder to construct summary latent variables by aggre- gating document latent variables. Subsequent re- search (Bražinskas et al., 2020; Iso et al., 2021) adopted a …

Content truncated to 500 words