YNU-HPCC at SemEval-2019 Task 9: Using a BERT and CNN-BiLSTM-GRU Model for Suggestion Mining

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1277–1281 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1277 YNU-HPCC at SemEval-2019 Task 9: Using a BERT and CNN-BiLSTM-GRU Model for Suggestion Mining Ping Yue, Jin Wang and Xuejie Zhang School of Information Science and Engineering Yunnan University Contact:Kunming, P.R. China xjzhang@ynu.edu.cn Abstract Consumer opinions towards commercial enti- ties are generally expressed through online re- views, blogs, and discussion forums. These opinions largely express positive and negative sentiments towards a given entity; however, they may also contain suggestions for improv- ing the entity. In this task, we extract sugges- tions from a given unstructured text, in con- trast to the traditional opinion mining system- s. This type suggestion mining is more ap- plicable and extends capabilities. In this pa- per, we propose the use of bidirectional en- coder representation learned from transform- ers (BERT) to address the problem of domain specific suggestion mining in task A. In detail, BERT is also used to extract feature vectors and perform fine-tuning for the task . For Task B, we applied an ensemble model to combine the BiLSTM, CNN, and GRU models, which can perform cross domain suggestion mining. Officially released results show that our sys- tem performs better than the baseline algorith- m does. 1 Introduction Suggestion mining is used to extract advice from text such as that provided in online reviews, blogs, discussion forums, and social media platform- s where consumers share their opinions toward- s commercial entities like brands, services, and products. Most of the traditional sentiment anal- ysis methods are emotion classifications. Opinion mining can improve service and quality. Such sys- tems have become an effective way for marketing, economics, politics, and advertising. The applica- tion of suggestion mining provides the motivation, for the SemEval 2019 Task 9 (Negi et al., 2019), which contains two subtasks that classify giv- en sentences into suggestion and non-suggestion classes. Subtask A requires a system to achieve domain specific training, whereby the test dataset will belong to the same domain as the training and development datasets. This was part of a sugges- tion forum for windows platform developers. Sub- task B applies the system to cross domain training, where training, development, and test datasets will belong to different domains. Training and devel- opment datasets will remain the same as Subtask A, while the test dataset will belong to the domain of hotel reviews. There are many methods in sentiment analy- sis. In many reports on this subject, it has been implied that these models help improve classifi- cation. Successful models include convolutional neural networks (CNN), long short-term memory (LSTM), and bi-directional LSTM (BiLSTM). C- NN can capture local n-gram features, while LST- M can maintain memory in the pipelines and solve the problem of long sequence dependence in neu- ral networks. In this paper, we propose a bidirectional en- coder representation learned from transformers (BERT) model (Devlin et al., 2018) for Task A. It comprises two phases. The …

Content truncated to 500 words