Hashtag Recommendation with Topical Attention-Based LSTM

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3019–3029, Osaka, Japan, December 11-17 2016. Hashtag Recommendation with Topical Attention-Based LSTM Yang Li†, Ting Liu†, Jing Jiang ‡, Liang Zhang † †Center for Social Computing and Information Retrieval, Harbin Institute of Technology, China ‡School of Information Systems, Singapore Management University, Singapore {yli,tliu,lzhang}@ir.hit.edu.cn jingjiang@smu.edu.sg Abstract Microblogging services allow users to create hashtags to categorize their posts. In recent years, the task of recommending hashtags for microblogs has been given increasing attention. However, most of existing methods depend on hand-crafted features. Motivated by the successful use of long short-term memory (LSTM) for many natural language processing tasks, in this paper, we adopt LSTM to learn the representation of a microblog post. Observing that hashtags indicate the primary topics of microblog posts, we propose a novel attention-based LSTM model which incorporates topic modeling into the LSTM architecture through an attention mechanism. We evaluate our model using a large real-world dataset. Experimental results show that our model significantly outperforms various competitive baseline methods. Furthermore, the incorporation of topical attention mechanism gives more than 7.4% improvement in F1 score compared with standard LSTM method. 1 Introduction Over the past few years, microblogging has experienced tremendous success and become very important as both a social network and a news media. There is a significant amount of information generated every day. To facilitate the navigation in the deluge of information, microblogging services allow users to insert hashtags starting with the “#” symbol (e.g., #followfriday) into their posts to indicate the context or the core idea. In this way, hashtags help bring together relevant microblogs on a particular topic or event and enhance information diffusion in microblog services. It has been proven that hashtags are important for many applications in microblogs (Efron, 2010; Bandyopadhyay et al., 2012; Davidov et al., 2010; Wang et al., 2011; Li et al., 2015). However, not all microblog posts have hashtags created by their authors. Reported in a recent study, only about 11% of tweets were annotated with one or more hashtags (Hong et al., 2012). Hence, the task of recommending hashtags for microblogs has become an important research topic and attracted much attention in recent years. Existing approaches to hashtag recommendation range from classification and collaborative filtering to probabilistic models such as naive Bayes and topic models. Most of these methods depend on sparse lex- ical features including bag-of-word (BoW) models and exquisitely designed patterns. However, feature engineering is labor-intensive and the sparse and discrete features cannot effectively encode semantic and syntactic information of words. On the other hand, neural models recently have shown great poten- tial for learning effective representations and delivered state-of-the-art performance on various natural language processing tasks (Cho et al., 2014; Tang et al., 2015; Rush et al., 2015). Among these methods, the long short-term memory (LSTM), a variant of recurrent neural network (RNN), is widely adopted due to its capability of capturing long-term dependencies in learning sequential representations (Hochreiter and Schmidhuber, 1997; …

Content truncated to 500 words