META: Metadata-Empowered Weak Supervision for Text Classification

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8351–8361, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8351 META: Metadata-Empowered Weak Supervision for Text Classification Dheeraj Mekala1 Xinyang Zhang2 Jingbo Shang1,3 1 Department of Computer Science and Engineering, University of California San Diego, CA, USA 2 Department of Computer Science, University of at Illinois Urbana-Champaign, IL, USA 3 Halıcıo˘glu Data Science Institute, University of California San Diego, CA, USA 1,3 {dmekala, jshang}@ucsd.edu 2 xz43@illinois.edu Abstract Recent advances in weakly supervised learn- ing enable training high-quality text classifiers by only providing a few user-provided seed words. Existing methods mainly use text data alone to generate pseudo-labels despite the fact that metadata information (e.g., author and timestamp) is widely available across var- ious domains. Strong label indicators exist in the metadata and it has been long overlooked mainly due to the following challenges: (1) metadata is multi-typed, requiring systematic modeling of different types and their combi- nations, (2) metadata is noisy, some metadata entities (e.g., authors, venues) are more com- pelling label indicators than others. In this paper, we propose a novel framework, META, which goes beyond the existing paradigm and leverages metadata as an additional source of weak supervision. Specifically, we organize the text data and metadata together into a text-rich network and adopt network motifs to capture appropriate combinations of metadata. Based on seed words, we rank and filter mo- tif instances to distill highly label-indicative ones as “seed motifs”, which provide addi- tional weak supervision. Following a boot- strapping manner, we train the classifier and expand the seed words and seed motifs itera- tively. Extensive experiments and case stud- ies on real-world datasets demonstrate supe- rior performance and significant advantages of leveraging metadata as weak supervision. 1 Introduction Weakly supervised text classification has recently gained much attention from the researchers because it reduces the burden of annotating the data. So far, the major source of weak supervision lies in text data itself (Agichtein and Gravano, 2000; Kuipers et al., 2006; Riloff et al., 2003; Tao et al., 2015; Meng et al., 2018; Mekala and Shang, 2020). These methods typically require a few user-provided seed Paper Authors Year Category P1 G. Hinton, S. Osindero, YW. Teh 2006 ML P2 G. Hinton, O. Vinyals, J. Dean 2015 ML P3 J. Dean, S.Ghemawat 2008 Sys (a) Examples of research papers with metadata. O. Vinyals G. Hinton 2016 neural system data learning J. Dean Doc Author 2 Author 1 P2 G. Hinton O. Vinyals P2 P1 … … … (b) A text-rich network view of the papers. (c) A motif pattern and a motif instance. Figure 1: Text corpus, text-rich network, and motif. words for each class as weak supervision. They ex- pand seed words with generated pseudo labels and improve their text classifier in an iterative fashion. Metadata information (e.g., author, published year) in addition to textual information, is widely available across various domains (e.g., news arti- cles, social media posts, and scientific papers) and it could …

Content truncated to 500 words