Reinforced Co-Training

Proceedings of NAACL-HLT 2018, pages 1252–1262 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Reinforced Co-Training Jiawei Wu Department of Computer Science University of California Santa Barbara, CA 93106 USA jiawei wu@cs.ucsb.edu Lei Li Toutiao AI Lab Bytedance Co. Ltd Beijing, 100080 China lileicc@gmail.com William Yang Wang Department of Computer Science University of California Santa Barbara, CA 93106 USA william@cs.ucsb.edu Abstract Co-training is a popular semi-supervised learning framework to utilize a large amount of unlabeled data in addition to a small la- beled set. Co-training methods exploit pre- dicted labels on the unlabeled data and se- lect samples based on prediction confidence to augment the training. However, the selec- tion of samples in existing co-training methods is based on a predetermined policy, which ig- nores the sampling bias between the unlabeled and the labeled subsets, and fails to explore the data space. In this paper, we propose a novel method, Reinforced Co-Training, to se- lect high-quality unlabeled samples to better co-train on. More specifically, our approach uses Q-learning to learn a data selection policy with a small labeled dataset, and then exploits this policy to train the co-training classifiers automatically. Experimental results on click- bait detection and generic text classification tasks demonstrate that our proposed method can obtain more accurate text classification re- sults. 1 Introduction Large labeled datasets are often required to obtain satisfactory performance for natural language pro- cessing tasks. However, it is time-consuming to la- bel text corpus manually. In the meanwhile, there are abundant unlabeled text corpora available on the web. Semi-supervised methods permit learn- ing improved supervised models by jointly train on a small labeled dataset and a large unlabeled dataset (Zhu, 2006; Chapelle et al., 2009). Co-training is one of the widely used semi- supervised methods, where two complementary classifiers utilize large amounts of unlabeled ex- amples to bootstrap the performance of each other iteratively (Blum and Mitchell, 1998; Nigam and Ghani, 2000). Co-training can be readily applied to NLP tasks since data in these tasks naturally Data Space Labeled Set Unlabeled Set Figure 1: Illustration of sample-selection issues in co- training methods. (1) Randomly sampled unlabeled ex- amples (2) will result in high sampling bias, which will cause bias shift towards the unlabeled dataset (←). (2) High-confidence examples (3) will contribute little during the model training, especially for discriminat- ing the boundary examples (△), resulting in myopic trained models. have two or more views, such as multi-lingual data (Wan, 2009) and document data (headline and content) (Ghani, 2000; Denis et al., 2003). In the co-training framework, each classifier is trained on one of the two views (aka a subset of features) of both labeled and unlabeled data, un- der the assumption that either view is sufficient to classify. In each iteration, the co-training al- gorithm selects high confidence samples scored by each of the classifiers to form an auto-labeled dataset, and the other classifier is then updated with both labeled data and additional auto-labeled set. However, as shown …

Content truncated to 500 words