Reinforced Sequence Training based Subjective Bias Correction

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 232–247 May 2-6, 2023 ©2023 Association for Computational Linguistics Friend-training: Learning from Models of Different but Related Tasks Mian Zhang†∗, Lifeng Jin⋄, Linfeng Song⋄, Haitao Mi⋄, Xiabing Zhou† and Dong Yu⋄ †Soochow University, Suzhou, China mzhang2@stu.suda.edu.cn, zhouxiabing@suda.edu.cn ⋄Tencent AI Lab, Bellevue, WA, USA {lifengjin,lfsong,haitaomi,dyu}@tencent.com Abstract Current self-training methods such as standard self-training, co-training, tri-training, and oth- ers often focus on improving model perfor- mance on a single task, utilizing differences in input features, model architectures, and train- ing processes. However, many tasks in natural language processing are about different but re- lated aspects of language, and models trained for one task can be great teachers for other related tasks. In this work, we propose friend- training, a cross-task self-training framework, where models trained to do different tasks are used in an iterative training, pseudo-labeling, and retraining process to help each other for better selection of pseudo-labels. With two dialogue understanding tasks, conversational semantic role labeling and dialogue rewriting, chosen for a case study, we show that the models trained with the friend-training frame- work achieve the best performance compared to strong baselines. 1 Introduction Many different machine learning algorithms, such as self-supervised learning (Mikolov et al., 2013; Devlin et al., 2019; Liu et al., 2021), semi- supervised learning (Yang et al., 2021) and weakly supervised learning (Zhou, 2018), aim at using un- labeled data to boost performance. They have been of even greater interest recently given the amount of unlabeled data available. Self-training (Scudder, 1965) is one semi-supervised learning mechanism aiming to improve model performance through pseudo-labeling and has been successfully ap- plied to computer vision (Lee et al., 2013; Chen et al., 2021), natural language processing (Dong and Schäfer, 2011; Bhat et al., 2021) and other fields (Wang et al., 2019; Kahn et al., 2020). The main challenge of self-training is how to select high-quality pseudo-labels. Current self- training algorithms mainly focus on a single task ∗Work done when interning at Tencent AI Lab. predicate: 喜欢 (like) arg-0: arg-1: 宫崎骏(Hayao Miyazaki) 知道久石让吗？ (Do you know Joe Hisaishi?) 我很喜欢他。 (I like him so much.) 知道啊，宫崎骏的很多电影配 乐都是久石让的，比如《幽灵 公主》。 (Yes, I do. Many of Hayao Miyazaki’s movie soundtracks are composed by Hisaishi, such as Princess Mononoke). cross-task supervision DR CSRL context current utterance rewritten utterance predicate-arguments 我很喜欢久石让。 (I like Joe Hisaishi so much.) Figure 1: An example of cross-task supervision between a CSRL parser and a DR system in a dialogue. 久石 让( Joe Hisaishi ) from the rewritten utterance provides cross-task supervision to 宫崎骏( Hayao Miyazaki ), the predicted arg-1 of 喜欢(like) from the CSRL parser, while 我( I ) to the predicted arg-0. when assessing the quality of pseudo-labels and suf- fer from gradual drifts of noisy instances (Zhang et al., 2021). This work is motivated by the observa- tion that learning targets of tasks represent different properties of the inputs, and some properties are shared across the tasks which can be used as …

Content truncated to 500 words