Leveraging Meta Information in Short Text Aggregation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4042–4049 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4042 Leveraging Meta Information in Short Text Aggregation He Zhao† Lan Du†∗ Guanfeng Liu‡ Wray Buntine† †Faculty of Information Technology Monash University, Australia ethanhezhao@gmail.com, {lan.du,wray.buntine}@monash.edu ‡Department of Computing Macquarie University, Australia guanfeng.liu@mq.edu.au Abstract Analysing topics in short texts (e.g., tweets and new headings) is a challenging task be- cause short texts often contain insufficient word co-occurrence information, which is im- portant to learn good topics in conventional topic topics. To deal with the insufficiency, we propose a generative model that aggregates short texts into clusters by leveraging the asso- ciated meta information. Our model can gen- erate more interpretable topics as well as doc- ument clusters. We develop an effective Gibbs sampling algorithm favoured by the fully lo- cal conjugacy in the model. Extensive exper- iments demonstrate that our model achieves better performance in terms of document clus- tering and topic coherence. 1 Introduction Texts generated on the internet (e.g., tweets, news headlines and product reviews) are usually short, which means that each individual document con- tains insufficient word co-occurrence information. Many existing topic models like Latent Dirichlet Allocation (LDA) (Blei et al., 2003) and its vari- ants infer topics purely based on the word occur- rence information, which often results in degraded performance and makes those models incapable of learning from short texts. Recently, many research efforts have been de- voted to analysing short texts. A common strategy is to aggregate short texts into clusters and then ap- ply topic models to those clusters. The clusters are expected to aggregate the word co-occurrence in- formation of the assigned documents. One widely- used option is known as self-aggregation, where we can aggregate short texts according to the con- textual information. For example, the contextual information of a document can be encoded by its topics so that the topic assignments can be used for ∗Corresponding author aggregation. This line of research includes models such as SATM (Quan et al., 2015), LTM (Li et al., 2018a), and PTM (Zuo et al., 2016a). On the other hand, many short texts, likes tweets, often come with meta information (meta-info for short, also known as meta-data or side information), such as authors, categories, hashtags, timestamps, etc. Therefore, another popular option is to aggregate short texts according to their meta-info. For exam- ple, we can assume that tweets published by the same users (Hong and Davison, 2010; Zhao et al., 2011) or with the same hashtags (Mehrotra et al., 2013) are likely to discuss similar topics. Those tweets can be aggregated into the same clusters. Although the above two aggregation schemes have yielded prominent results on short text anal- ysis, there is still space for improvement. For ex- ample, in tweet analysis: if we ignore the associ- ated meta-info as in the self-aggregation scheme, we may lose important information; on the other hand, it may not be a perfect …

Content truncated to 500 words