Zoho at SemEval-2019 Task 9: Semi-supervised Domain Adaptation using Tri-training for Suggestion Mining

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1282–1286 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1282 Zoho at SemEval-2019 Task 9: Semi-supervised Domain Adaptation using Tri-training for Suggestion Mining Sai Prasanna Zoho saiprasanna.r@zohocorp.com sai.r.prasanna@gmail.com Sri Ananda Seelan Zoho anandaseelan.ln@zohocorp.com Abstract This paper describes our submission for the SemEval-2019 Suggestion Mining task. simple Convolutional Neural Network (CNN)A classifier with contextual word representations from a pre-trained language model is used for sentence classification. The model is trained using tri-training, a semi-supervised bootstrapping mechanism for labelling unseen data. Tri-training proved to be an effective technique to accommodate domain shift for cross-domain suggestion mining (Subtask B) where there is no hand labelled training data. For in-domain evaluation (Subtask A), we use the same technique to augment the training set. Our system ranks thirteenth in Subtask A with an F1-score of 68.07 and third in Subtask B with an F1-score of 81.94. 1 Introduction Task 9 of SemEval-2019 (Negi et al., 2019) fo- cuses on mining sentences that contain sugges- tions in online discussions and reviews. Sugges- tion Mining is modelled as a sentence classifica- tion task with two Subtasks: • Subtask A evaluates the classifier perfor- mance on a technical domain specific setting. • Subtask B evaluates the domain adaptability of a model by doing cross-domain suggestion classification on hotel reviews. We approached this task as an opportunity to test the effectiveness of transfer learning and semi- supervised learning techniques. In Subtask A, the high class imbalance and relatively smaller size of the training data made it an ideal setup for evaluating the efficacy of recent transfer learn- ing techniques. Using pre-trained language mod- els for contextual word representations has been shown to improve many Natural Language Pro- cessing (NLP) tasks (Peters et al., 2018; Ruder and Howard, 2018; Radford, 2018; Devlin et al., 2018). This transfer learning technique is also an effective method when less labelled data is available as shown in (Ruder and Howard, 2018). In this work, we use the BERT model (Devlin et al., 2018) for obtaining contextual representa- tions. This results in enhanced scores even for simple baseline classifiers. Subtask B requires the system to not use man- ually labelled data and hence it lends itself to a classic semi-supervised learning scenario. Many methods have been proposed for domain adapta- tion for NLP (Blitzer et al., 2007; Chen et al., 2011; Chen and Cardie, 2018; Zhou and Li, 2005; Blum and Mitchell, 1998). We use a label boot- strapping technique called tri-training (Zhou and Li, 2005) with which unlabelled samples are la- belled iteratively with increasing confidence at each training iteration(explained in Section 2.4). Ruder and Plank (2018) shows the effectiveness of tri-training for baseline deep neural models in text classification under domain shift. They also pro- pose a multi-task approach for tri-training, how- ever we only adapt the classic tri-training proce- dure presented for suggestion mining task. Detailed explanation of the submitted system and experiments are elicited in the following …

Content truncated to 500 words