Structural Constraints and Natural Language Inference for End-to-End Flowchart Grounded Dialog Response Generation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1107 - 1119 December 7-11, 2022 ©2022 Association for Computational Linguistics Zero-Shot Text Classification with Self-Training Ariel Gera∗, Alon Halfon∗, Eyal Shnarch∗, Yotam Perlitz∗, Liat Ein-Dor, Noam Slonim IBM Research {ariel.gera1, yotam.perlitz}@ibm.com, {alonhal, eyals, liate, noams}@il.ibm.com Abstract Recent advances in large pretrained language models have increased attention to zero-shot text classification. In particular, models fine- tuned on natural language inference datasets have been widely adopted as zero-shot clas- sifiers due to their promising results and off- the-shelf availability. However, the fact that such models are unfamiliar with the target task can lead to instability and performance issues. We propose a plug-and-play method to bridge this gap using a simple self-training approach, requiring only the class names along with an unlabeled dataset, and without the need for do- main expertise or trial and error. We show that fine-tuning the zero-shot classifier on its most confident predictions leads to significant per- formance gains across a wide range of text clas- sification tasks, presumably since self-training adapts the zero-shot model to the task at hand. 1 Introduction Large language models have revolutionized the field of natural language processing, leading to great leaps in performance across the NLP task landscape (Devlin et al., 2018; Raffel et al., 2020; Brown et al., 2020). The pretrain-finetune paradigm has led to a significant reduction in the amount of labeled data required for obtaining high performance on downstream tasks. However, the need to collect labeled examples for each target task remains an obstacle, limiting the usage of language models in practice, at scale. Thus, the more ambitious vision of a general- purpose zero-shot model – one that can tackle many different tasks without requiring labeled data – has become an enticing goal for the community. This notion is increasingly gaining attention, with recent works suggesting new paradigms that aim to utilize the language understanding capabilities of large models for the zero-shot scenario. ∗These authors contributed equally to this work. In their pioneering work on more general- purpose zero-shot models, Yin et al. (2019) pro- pose to formulate text classification tasks as a tex- tual entailment problem (Dagan et al., 2005). This mapping enables using a model trained on natural language inference (NLI) as a zero-shot text classi- fier for a wide variety of unseen downstream tasks. The underlying idea is fairly intuitive. To deter- mine if a particular text should be assigned to, e.g., the "sports" class or the "politics" class, one con- structs sentences such as "This text is about sports" and "This text is about politics", respectively; the model prediction as to which one is most entailed by the original text can then be used to determine the predicted class label. Similarly, some recent works have tried to map even more varied types of NLP tasks into a unified cross-task format (Wei et al., 2022; Zhong et al., 2021; Bragg et al., 2021; Sanh et al., 2022). Such unified task formats en- able …

Content truncated to 500 words