Weakly-supervised Text Classification Based on Keyword Graph

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2803–2813 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2803 Weakly-supervised Text Classification Based on Keyword Graph Lu Zhang1∗ Jiandong Ding2 Yi Xu1∗ Yingyao Liu2 Shuigeng Zhou1† 1Shanghai Key Lab of Intelligent Information Processing, and School of Computer Science, Fudan University, China 2Alibaba Group {l_zhang19, jdding, yxu17, sgzhou}@fudan.edu.cn Abstract Weakly-supervised text classification has re- ceived much attention in recent years for it can alleviate the heavy burden of annotating mas- sive data. Among them, keyword-driven meth- ods are the mainstream where user-provided keywords are exploited to generate pseudo- labels for unlabeled texts. However, existing methods treat keywords independently, thus ignore the correlation among them, which should be useful if properly exploited. In this paper, we propose a novel framework called ClassKG to explore keyword-keyword correla- tion on keyword graph by GNN. Our frame- work is an iterative process. In each itera- tion, we first construct a keyword graph, so the task of assigning pseudo labels is trans- formed to annotating keyword subgraphs. To improve the annotation quality, we introduce a self-supervised task to pretrain a subgraph an- notator, and then finetune it. With the pseudo labels generated by the subgraph annotator, we then train a text classifier to classify the unlabeled texts. Finally, we re-extract key- words from the classified texts. Extensive experiments on both long-text and short-text datasets show that our method substantially outperforms the existing ones. 1 Introduction Text classification is one of the most fundamen- tal tasks in natural language processing (NLP). In real-world scenarios, labeling massive texts is time- consuming and expensive, especially in some spe- cific areas that need domain experts to participate. Weakly-supervised text classification (WTC) has received much attention in recent years because it can substantially reduce the workload of anno- tating massive data. Among the existing methods, the mainstream form is keyword-driven (Agichtein and Gravano, 2000; Riloff et al., 2003; Kuipers ∗The author did most work during internship at Alibaba. †Correspondence author Text%A:%%𝒘𝒊𝒏𝒅𝒐𝒘𝒔 𝑖𝑠 𝑎 𝑝𝑟𝑜𝑑𝑢𝑐𝑡 𝑜𝑓 𝒎𝒊𝒄𝒓𝒐𝒔𝒐𝒇𝒕, … … Text B: 𝑠𝑡𝑖𝑐𝑘𝑠 ℎ𝑖𝑠 ℎ𝑒𝑎𝑑 𝑜𝑢𝑡 𝒄𝒂𝒓 𝒘𝒊𝒏𝒅𝒐𝒘𝒔, 𝑡ℎ𝑒 𝒘𝒊𝒏𝒅𝒐𝒘𝒔 … 𝑤𝑖𝑛𝑑𝑜𝑤𝑠 𝑑𝑟𝑖𝑣𝑒𝑟 𝑚𝑖𝑐𝑟𝑜𝑠𝑜𝑓𝑡 𝑐𝑎𝑟 𝑚𝑢𝑠𝑡𝑎𝑛𝑔 𝑏𝑎𝑠𝑒𝑏𝑎𝑙𝑙 ℎ𝑜𝑐𝑘𝑒𝑦 𝑏𝑖𝑘𝑒 𝐺𝑁𝑁 𝑇𝑒𝑥𝑡 𝐴 𝑇𝑒𝑥𝑡 𝐵 “𝑐𝑜𝑚𝑝𝑢𝑡𝑒𝑟” “𝑡𝑟𝑎𝑓𝑓𝑖𝑐” 𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑒 𝑝𝑠𝑒𝑢𝑑𝑜 𝑙𝑎𝑏𝑒𝑙s “𝑐𝑜𝑚𝑝𝑢𝑡𝑒𝑟” “𝑐𝑜𝑚𝑝𝑢𝑡𝑒𝑟” 𝑐𝑜𝑢𝑛𝑡 𝑐𝑜𝑢𝑛𝑡 (𝑎) (𝑏) Figure 1: (a) Existing methods do not consider the cor- relation among keywords, which will generate wrong pseudo-label for text B. (b) Our method exploits the correlation among keywords by GNN over a keyword- graph, and converts the task of assigning pseudo-labels for unlabeled texts to annotating subgraphs, which leads to much better performance. et al., 2006; Meng et al., 2018, 2019, 2020; Mekala and Shang, 2020; Wang et al., 2021; Shen et al., 2021), where the users need only to provide some keywords for each class. Such class-relevant key- words are then used to generate pseudo-labels for unlabeled texts. Keyword-driven methods usually follow an it- erative process: generating pseudo-labels using keywords, building a text classifier, and updat- ing the keywords or self-training the …

Content truncated to 500 words