Semi-Supervised Data Programming with Subset Selection

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4640–4651 August 1–6, 2021. ©2021 Association for Computational Linguistics 4640 Semi-Supervised Data Programming with Subset Selection Ayush Maheshwari 1, Oishik Chatterjee 1, Krishnateja Killamsetty 2, Ganesh Ramakrishnan 1 , Rishabh Iyer 2 1Department of CSE, IIT Bombay, India 2 The University of Texas at Dallas {ayusham, oishik, ganesh}@cse.iitb.ac.in {krishnateja.killamsetty, rishabh.iyer}@utdallas.edu Abstract The paradigm of data programming, which uses weak supervision in the form of rules/labelling functions, and semi-supervised learning, which augments small amounts of labelled data with a large unlabelled dataset, have shown great promise in several text classi- fication scenarios. In this work, we argue that by not using any labelled data, data program- ming based approaches can yield sub-optimal performances, particularly when the labelling functions are noisy. The first contribution of this work is an introduction of a framework, SPEAR which is a semi-supervised data pro- gramming paradigm that learns a joint model that effectively uses the rules/labelling func- tions along with semi-supervised loss func- tions on the feature space. Next, we also study SPEAR-SS which additionally does subset se- lection on top of the joint semi-supervised data programming objective and selects a set of examples that can be used as the labelled set by SPEAR. The goal of SPEAR-SS is to ensure that the labelled data can comple- ment the labelling functions, thereby benefit- ing from both data-programming as well as ap- propriately selected data for human labelling. We demonstrate that by effectively combining semi-supervision, data-programming, and sub- set selection paradigms, we significantly out- perform the current state-of-the-art on seven publicly available datasets. 1 1 Introduction Modern machine learning techniques rely on large amounts of labelled training data for text clas- sification tasks such as spam detection, (movie) genre classification, sequence labelling, etc. Super- vised learning approaches have utilised such large amounts of labelled data and, this has resulted in 1The source code is available at https://github.com/ayushbits/ Semi-Supervised-LFs-Subset-Selection huge successes in the last decade. However, the acquisition of labelled data, in most cases, entails a painstaking process requiring months of human effort. Several techniques such as active learning, distant supervision, crowd-consensus learning, and semi-supervised learning have been proposed to reduce the annotation cost (Settles et al., 2008). However, clean annotated labels continue to be crit- ical for reliable results (Bach et al., 2019; Goh et al., 2018). Recently, Ratner et al. (2016) proposed a paradigm on data-programming in which several Labelling Functions (LF) written by humans are used to weakly associate labels with the instances. In data programming, users encode the weak su- pervision in the form of labelling functions. On the other hand, traditional semi-supervised learning methods combine a small amount of labelled data with large unlabelled data (Kingma et al., 2014). In this paper, we leverage semi-supervision in the feature space for more effective data programming using labelling functions. 1.1 Motivating Example We illustrate the LFs on one of the seven tasks on which we experiment with, viz., identifying spam/no-spam comments in the YouTube reviews. …

Content truncated to 500 words