Text Classification with Few Examples using Controlled Generalization

Proceedings of NAACL-HLT 2019, pages 3158–3167 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3158 Text Classification with Few Examples using Controlled Generalization Abhijit Mahabal Google, Pinterest amahabal@ gmail.com Jason Baldridge Google jridge@ google.com Burcu Karagol Ayan Google burcuka@ google.com Vincent Perot Google vperot@ google.com Dan Roth UPenn danroth@ seas.upenn.edu Abstract Training data for text classification is often limited in practice, especially for applications with many output classes or involving many related classification problems. This means classifiers must generalize from limited evi- dence, but the manner and extent of general- ization is task dependent. Current practice pri- marily relies on pre-trained word embeddings to map words unseen in training to similar seen ones. Unfortunately, this squishes many components of meaning into highly restricted capacity. Our alternative begins with sparse pre-trained representations derived from unla- beled parsed corpora; based on the available training data, we select features that offers the relevant generalizations. This produces task- specific semantic vectors; here, we show that a feed-forward network over these vectors is es- pecially effective in low-data scenarios, com- pared to existing state-of-the-art methods. By further pairing this network with a convolu- tional neural network, we keep this edge in low data scenarios and remain competitive when using full training sets. 1 Introduction Modern neural networks are highly effective for text classification, with convolutional neural net- works (CNNs) as the de facto standard for clas- sifiers that represent both hierarchical and order- ing information implicitly in a deep network (Kim, 2014). Deep models pre-trained on language model objectives and fine-tuned to available train- ing data have recently smashed benchmark scores on a wide range of text classification problems (Peters et al., 2018; Howard and Ruder, 2018; De- vlin et al., 2018). Despite the strong performance of these ap- proaches for large text classification datasets, chal- lenges still arise with small datasets with few, pos- sibly imbalanced, training examples per class. La- bels can be obtained cheaply from crowd workers for some languages, but there are a nearly unlim- ited number of bespoke, challenging text classifi- cation problems that crop up in practical settings (Yu et al., 2018). Obtaining representative labeled examples for classification problems with many labels, like taxonomies, is especially challenging. Text classification is a broad but useful term and covers classification based on topic, on sentiment, and even social status. As Systemic Functional Linguists such as Halliday (1985) point out, lan- guage carries many kinds of meanings. For exam- ple, words such as ambrosial and delish inform us not just of the domain of the text (food) and sen- timent, but perhaps also of the age of the speaker. Text classification problems differ on the dimen- sions they distinguish along and thus in the words that help in identifying the class. As Sachan et al. (2018) show, classifiers mostly focus on sub-lexicons; they memorize patterns in- stead of extending more general knowledge about language to a particular task. When there is low lexical overlap between training and …

Content truncated to 500 words