Yes, No or IDK: The Challenge of Unanswerable Yes/No Questions

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 99 - 106 July 10-15, 2022 ©2022 Association for Computational Linguistics LEA: Meta Knowledge-Driven Self-Attentive Document Embedding for Few-Shot Text Classification S. K. Hong and Tae Young Jang Samsung SDS, ML Research Center {s.k.hong, tae10.jang}@samsung.com Abstract In recent years, NLP has advanced greatly along with the proliferation of pre-trained lan- guage models. The pre-trained language mod- els are also properly adapted to downstream tasks when there is sufficient labeled data. However, in real-world applications, we often encounter the deficiency of labeled data. When only given a few instances for a new task, ex- tracting task-aware features from a pre-trained language model regardless of the adaptation is a promising alternative. In the study, we pro- pose a novel embedding transfer method, called LEA, for leveraging pre-trained language mod- els with even only few-shot instances. LEA derives meta-level attention aspects using our new meta-learning framework. We evaluate our method on five text classification bench- mark datasets. The results show that the novel method robustly provides the competitive per- formance compared to recent few-shot learning methods. 1 Introduction A deficiency of supervised data is often experi- enced in real-world NLP applications. Few-shot learning aims to yield an AI-driven NLP model capable of recognizing unseen tasks using a few labeled data. Meanwhile, fine-tuning pre-trained models (PTMs) (Howard and Ruder, 2018; Devlin et al., 2019; Lan et al., 2019; Liu et al., 2019) has been the most successful approach in recent years of NLP. Unfortunately, it is still challenging to uti- lize PTMs (Lee et al., 2019) in few-shot learning. To address this subtle problem (Sun et al., 2019), we propose a meta-knowledge driven self-attentive embedding transfer method, called LEA (LEarning- to-Attend), based on a novel meta-learning frame- work, through which meta-level attention aspects are derived by encoding how to attend for given tasks. LEA is an efficient and practical method that facilitates the utilization of large-sized PTMs in few-shot learning. There are the two common transfer learning paradigms in NLP: feature-based transfer (Cer et al., 2018) and fine-tuning (Houlsby et al., 2019). Our approach belongs to the feature-based transfer. LEA includes two key ideas: (1) construction of a meta-level attention aspects dictionary and (2) in- ference of the task-specific attention aspects upon the arrival of a new task. The former is a process by which useful meta-level attention aspects across tasks are derived based on a particular PTM via our meta-learning framework. The latter refers to as a task-adaption process, where a subset of task- specific attention aspects is inferred by determining the top-k most relevant attention aspects from the meta-level attention aspects dictionary. While LEA can be applied to a wide variety of downstream tasks, we demonstrate LEA on few-shot text classi- fication problems in the paper. 2 Related Work Few-shot text classification: In (Geng et al., 2019), INDUCTION is proposed to build class- wise embedding to represent each class using a particular …

Content truncated to 500 words