An Unsupervised Neural Attention Model for Aspect Extraction

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 388–397 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1036 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 388–397 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1036 An Unsupervised Neural Attention Model for Aspect Extraction Ruidan He†‡, Wee Sun Lee†, Hwee Tou Ng†, and Daniel Dahlmeier‡ †Department of Computer Science, National University of Singapore ‡SAP Innovation Center Singapore †{ruidanhe,leews,nght}@comp.nus.edu.sg ‡d.dahlmeier@sap.com Abstract Aspect extraction is an important and chal- lenging task in aspect-based sentiment analysis. Existing works tend to ap- ply variants of topic models on this task. While fairly successful, these methods usually do not produce highly coherent as- pects. In this paper, we present a novel neural approach with the aim of discov- ering coherent aspects. The model im- proves coherence by exploiting the distri- bution of word co-occurrences through the use of neural word embeddings. Unlike topic models which typically assume in- dependently generated words, word em- bedding models encourage words that ap- pear in similar contexts to be located close to each other in the embedding space. In addition, we use an attention mech- anism to de-emphasize irrelevant words during training, further improving the co- herence of aspects. Experimental results on real-life datasets demonstrate that our approach discovers more meaningful and coherent aspects, and substantially outper- forms baseline methods on several evalua- tion tasks. 1 Introduction Aspect extraction is one of the key tasks in senti- ment analysis. It aims to extract entity aspects on which opinions have been expressed (Hu and Liu, 2004; Liu, 2012). For example, in the sentence “The beef was tender and melted in my mouth”, the aspect term is “beef”. Two sub-tasks are per- formed in aspect extraction: (1) extracting all as- pect terms (e.g., “beef”) from a review corpus, (2) clustering aspect terms with similar meaning into categories where each category represents a single aspect (e.g., cluster “beef”, “pork”, “pasta”, and “tomato” into one aspect food). Previous works for aspect extraction can be cat- egorized into three approaches: rule-based, super- vised, and unsupervised. Rule-based methods usu- ally do not group extracted aspect terms into cate- gories. Supervised learning requires data annota- tion and suffers from domain adaptation problems. Unsupervised methods are adopted to avoid re- liance on labeled data needed for supervised learn- ing. In recent years, Latent Dirichlet Allocation (LDA) (Blei et al., 2003) and its variants (Titov and McDonald, 2008; Brody and Elhadad, 2010; Zhao et al., 2010; Mukherjee and Liu, 2012) have become the dominant unsupervised approach for aspect extraction. LDA models the corpus as a mixture of topics (aspects), and topics as distri- butions over word types. While the mixture of aspects discovered by LDA-based models may de- scribe a corpus fairly well, we find that the individ- ual aspects inferred are of poor quality – aspects often consist of unrelated or loosely-related con- cepts. This may substantially …

Content truncated to 500 words