Beyond Possession Existence: Duration and Co-Possession

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 800–806 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 800 Tree-Structured Neural Topic Model Masaru Isonuma1 Junichiro Mori1 Danushka Bollegala2 Ichiro Sakata1 1 The University of Tokyo 2 University of Liverpool {isonuma, isakata}@ipr-ctr.t.u-tokyo.ac.jp mori@mi.u-tokyo.ac.jp danushka@liverpool.ac.uk Abstract This paper presents a tree-structured neural topic model, which has a topic distribution over a tree with an infinite number of branches. Our model parameterizes an unbounded ances- tral and fraternal topic distribution by applying doubly-recurrent neural networks. With the help of autoencoding variational Bayes, our model improves data scalability and achieves competitive performance when inducing latent topics and tree structures, as compared to a prior tree-structured topic model (Blei et al., 2010). This work extends the tree-structured topic model such that it can be incorporated with neural models for downstream tasks. 1 Introduction Probabilistic topic models, such as latent Dirich- let allocation (LDA; Blei et al., 2003), are applied to numerous tasks including document modeling and information retrieval. Recently, Srivastava and Sutton (2017); Miao et al. (2017) have applied the autoencoding variational Bayes (AEVB; Kingma and Welling, 2014; Rezende et al., 2014) frame- work to basic topic models such as LDA. AEVB improves data scalability in conventional models. The limitation of the basic topic models is that they induce topics as flat structures, not organizing them into coherent groups or hierarchies. Tree- structured topic models (Griffiths et al., 2004), which detect the latent tree structure of topics, can overcome this limitation. These models induce a tree with an infinite number of nodes and assign a generic topic to the root and more detailed topics to the leaf nodes. In Figure 1, we show an exam- ple of topics induced by our model. Such char- acteristics are preferable for several downstream tasks, such as document retrieval (Weninger et al., 2012), aspect-based sentiment analysis (Kim et al., 2013) and extractive summarization (Celikyilmaz Root Carry Purchase Cover 1: quality months zipper time back 11: sleeve inside inch protection nice 111: bottom cover top plastic scratches 112: color cover mac keyboard love 12: perfect quality price bought size 121: item return receive amazon money 122: price recommend buy perfect love 13: pockets carry strap shoulder compartment 131: big laptops tablet description hp 132: books school carry bags back Figure 1: Topics inferred by our tree-structured topic model from Amazon reviews of laptop bags. The five most frequent words are shown and manually labeled. and Hakkani-Tur, 2010), because they provide suc- cinct information from multiple viewpoints. For instance, in the case of document retrieval of prod- uct reviews, some users are interested in the general opinions about bag covers, while others pay more attention to specific topics such as the hardness or color of the covers. The tree structure can navigate users to the documents with desirable granularity. However, it is difficult to use tree-structured topic models with neural models for downstream tasks. While neural models require a large amount of data for …

Content truncated to 500 words