Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3091–3101 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3091 Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification Yaqing Wang1, Song Wang1,2, Quanming Yao∗3, Dejing Dou1 1Baidu Research, Baidu Inc., China 2 Department of ECE, University of Virginia, USA 3 Department of EE, Tsinghua University, China {wangyaqing01, v_wangsong07, doudejing}@baidu.com qyaoaa@tsinghua.edu.cn Abstract Short text classification is a fundamental task in natural language processing. It is hard due to the lack of context information and labeled data in practice. In this paper, we propose a new method called SHINE, which is based on graph neural network (GNN), for short text classification. First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs which introduce more semantic and syntactic information. Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts. Thus, comparing with existing GNN-based methods, SHINE can better exploit interac- tions between nodes of the same types and cap- ture similarities between short texts. Extensive experiments on various benchmark short text datasets show that SHINE consistently outper- forms state-of-the-art methods, especially with fewer labels.1 1 Introduction Short texts such as tweets, news feeds and web search snippets appear daily in our life (Pang and Lee, 2005; Phan et al., 2008). To understand these short texts, short text classification (STC) is a fundamental task which can be found in many applications such as sentiment analysis (Chen et al., 2019), news classification (Yao et al., 2019) and query intent classification (Wang et al., 2017). STC is particularly hard in comparison to long text classification due to two key issues. The first key issue is that short texts only contain one or a few sentences whose overall length is small, which lack enough context information and strict syntactic structure to understand the meaning of texts (Tang et al., 2015; Wang et al., 2017). For example, it is hard to get the meaning of "Birthday girl is an amusing ride" without 1Codes are available at https://github.com/tata1661/ SHINE-EMNLP21. knowing "Birthday girl" is a 2001 movie. harder case is to understand a web search snippetA such as "how much Tesla", which usually does not contain word order nor function words (Phan et al., 2008). In addition, real STC tasks usually only have a limited number of labeled data compared to the abundant unlabeled short texts emerging everyday (Hu et al., 2019). Therefore, auxiliary knowledge is required to understand short texts, examples include concepts that can be found in common sense knowledge graphs (Wang et al., 2017; Chen et al., 2019), latent topics extracted from the short text dataset (Hu et al., 2019), and entities residing in knowledge graphs (Hu et al., 2019). However, simply enriching auxiliary knowledge cannot solve the shortage of labeled data, which is another key issue commonly faced by real STC tasks (Pang and Lee, 2005; Phan et al., 2008). Yet the popularly used deep models require large-scale …

Content truncated to 500 words