Unsupervised Label Refinement Improves Dataless Text Classification

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4165–4178 August 1–6, 2021. ©2021 Association for Computational Linguistics 4165 Unsupervised Label Refinement Improves Dataless Text Classification Zewei Chu1 Karl Stratos2 Kevin Gimpel3 1The University of Chicago, 5730 S Ellis Ave, Chicago, IL 60637, USA 2Rutgers University, 110 Frelinghuysen Road, Piscataway, NJ 08854, USA 3Toyota Technological Institute at Chicago, 6045 S Kenwood Ave, Chicago, IL 60637, USA zeweichu@uchicago.edu, karlstratos@gmail.com, kgimpel@ttic.edu Abstract Dataless text classification is capable of classi- fying documents into previously unseen labels by assigning a score to any document paired with a label description. While promising, it crucially relies on accurate descriptions of the label set for each downstream task. This re- liance causes dataless classifiers to be highly sensitive to the choice of label descriptions and hinders the broader application of data- less classification in practice. In this paper, we ask the following question: how can we im- prove dataless text classification using the in- puts of the downstream task dataset? Our pri- mary solution is a clustering based approach. Given a dataless classifier, our approach re- fines its set of predictions using k-means clus- tering. We demonstrate the broad applicabil- ity of our approach by improving the perfor- mance of two widely used classifier architec- tures, one that encodes text-category pairs with two independent encoders and one with a sin- gle joint encoder. Experiments show that our approach consistently improves dataless clas- sification across different datasets and makes the classifier more robust to the choice of label descriptions.1 1 Introduction Dataless text classification aims at classifying text into categories without using any annotated training data from the task of interest. Prior work (Chang et al., 2008; Song and Roth, 2014) has shown that with effective ways to represent texts and labels, dataless classifiers can perform text classification on unbounded label sets if suitable descriptions of the labels are provided. There have been many previous efforts in data- less or zero-shot text classification (Dauphin et al., 2013; Nam et al., 2016; Li et al., 2016; Ma et al., 1Code and data available at https://github.com/ ZeweiChu/ULR. 2016; Shu et al., 2017; Fei and Liu, 2016; Zhang et al., 2019; Yogatama et al., 2017; Mullenbach et al., 2018; Rios and Kavuluru, 2018; Meng et al., 2019). Several settings have been considered across this prior work, and some have used slightly different definitions of dataless classifiers. In this paper, we use the term “dataless text classification” to refer to methods that: (1) can assign scores to any document-category pair, and (2) do not require any annotated training data from downstream tasks. A dataless classifier can therefore be immediately adapted to a particular label set in a downstream task dataset by scoring each possible label for a document and returning the label with the highest score. Dataless classifiers are typically built from large-scale freely available text resources such as Wikipedia (Chang et al., 2008; Yin et al., 2019). A well known problem with dataless classifiers is that the choice of label …

Content truncated to 500 words