Simplified Graph Learning for Inductive Short Text Classification

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10717 - 10724 December 7-11, 2022 ©2022 Association for Computational Linguistics Simplified Graph Learning for Inductive Short Text Classification Kaixin Zheng∗1,2, Yaqing Wang∗1, Quanming Yao3, Dejing Dou1 1Baidu Research, Baidu Inc., China 2 Department of SIOE, Beihang University, China 3 Department of EE, Tsinghua University, China zhengkaixin@buaa.edu.cn {wangyaqing01, doudejing}@baidu.com qyaoaa@tsinghua.edu.cn Abstract Short text classification (STC) is hard as short texts lack context information and labeled data is not enough. Graph neural networks ob- tain the state-of-the-art on STC since they can merge various auxiliary information via the message passing framework. However, exist- ing works conduct transductive learning, which requires retraining to accommodate new sam- ples and takes large memory. In this paper, we present SimpleSTC which handles inductive STC problem but only leverages words. We construct word graph from an external large corpus to compensate for the lack of semantic information, and learn text graph to handle the lack of labeled data. Results show that Sim- pleSTC obtains state-of-the-art performance with lower memory consumption and faster in- ference speed. 1 1 Introduction Short text classification (STC) is a fundamental task in many applications, such as sentiment analy- sis (Chen et al., 2019) and query intent classifica- tion (Wang et al., 2017). Due to the limited length, short texts lack context and strict syntactic struc- ture, which makes them hard to understand (Wang et al., 2017). Hence, existing STC methods strive to enrich short texts by incorporating various aux- iliary information, including concepts of external knowledge bases (Wang et al., 2017; Chen et al., 2019; Wang et al., 2022), topics discovered in the corpus (Zeng et al., 2018; Yang et al., 2021), part- of-speech (POS) tags (Wang et al., 2021), and enti- ties residing in knowledge graphs (Hu et al., 2019). Another challenge faced by STC is the shortage of labeled data in practice (Pang and Lee, 2005; Phan et al., 2008). Hence, recent STC methods (Hu et al., 2019; Wang et al., 2021; Yang et al., ∗ Equal contribution. K. Zheng did his work during internship at Baidu Research. Correspondence to: Y. Wang. 1Codes are available at https://github.com/tata1661/ SimpleSTC-EMNLP22. 2021) further consider semi-supervised STC prob- lem where only a small portion of the corpus is labeled. The state-of-the-art STC models are based on graph neural networks (GNNs) (Defferrard et al., 2016; Kipf and Welling, 2016). HGAT (Hu et al., 2019) applies a GNN with dual-level attention on a corpus-level graph containing topics, entities and texts. SHINE (Wang et al., 2021) designs a hier- archical GNN to operate on a corpus-level hierar- chical graph of word-level components including words, part-of-speech (POS) tags and entities, and a learned text graph. In summary, they conduct trans- ductive learning, modeling the whole dataset (in- cluding the training and testing samples) as a large heterogeneous graph of various semantic concepts. Hence, they both suffer from two problems: (i) re- quire model retraining if new samples come and (ii) have high memory consumption …

Content truncated to 500 words