Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4821–4830, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4821 Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification Linmei Hu1, Tianchi Yang1, Chuan Shi*1, Houye Ji1, Xiaoli Li2 1Beijing University of Posts and Telecommunications, China 2Institute for Infocomm Research, Singapore {hulinmei,yangtianchi,shichuan,jhy1993}@bupt.edu.cn xlli@i2r.a-star.edu.sg Abstract Short text classification has found rich and critical applications in news and tweet tagging to help users find relevant information. Due to lack of labeled training data in many practical use cases, there is a pressing need for study- ing semi-supervised short text classification. Most existing studies focus on long texts and achieve unsatisfactory performance on short texts due to the sparsity and limited labeled data. In this paper, we propose a novel hetero- geneous graph neural network based method for semi-supervised short text classification, leveraging full advantage of few labeled data and large unlabeled data through information propagation along the graph. In particular, we first present a flexible HIN (heterogeneous information network) framework for model- ing the short texts, which can integrate any type of additional information as well as cap- ture their relations to address the semantic sparsity. Then, we propose Heterogeneous Graph ATtention networks (HGAT) to embed the HIN for short text classification based on a dual-level attention mechanism, including node-level and type-level attentions. The at- tention mechanism can learn the importance of different neighboring nodes as well as the im- portance of different node (information) types to a current node. Extensive experimental results have demonstrated that our proposed model outperforms state-of-the-art methods across six benchmark datasets significantly. 1 Introduction With the rapid development of online social me- dia and e-commerce, short texts, such as online news, queries, reviews, tweets, are increasingly widespread on the Internet (Song et al., 2014). Short text classification can be widely applied in many domains, ranging from sentiment analysis to news tagging/categorization and query intent classification (Aggarwal and Zhai, 2012; Meng et al., 2018). In many practical scenarios, the labeled data is scarce, while human labeling is time-consuming and may require expert knowl- edge (Aggarwal and Zhai, 2012). As a conse- quence, there is a pressing need for studying semi- supervised short text classification with a rela- tively small number of labeled training data. Nevertheless, semi-supervised short text clas- sification is nontrivial due to the following chal- lenges. Firstly, short texts are usually seman- tically sparse and ambiguous, lacking contexts (Phan et al., 2008). While some methods have been proposed to incorporate additional informa- tion such as entities (Wang et al., 2013, 2017), they are unable to consider the relational data such as the semantic relations among entities. Sec- ondly, the labeled training data is limited, which leads to traditional and neural supervised meth- ods (Wang and Manning, 2012; Kim, 2014; Zhang et al., 2015) ineffective. As such, how to make full use of the limited labeled data and large number of …

Content truncated to 500 words