Lijunyi at SemEval-2019 Task 9: An attention-based LSTM and ensemble of different models for suggestion mining from online reviews and forums

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1208–1212 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1208 Lijunyi at SemEval-2019 Task 9: An attention-based LSTM model and ensemble of different models for suggestion mining from online reviews and forums Junyi Li, Haiyan Ding∗ School of Information Science and Engineering Yunnan University, Yunnan, P.R. China ∗Corresponding author: hyding@ynu.edu.cn Abstract In this paper, we describe a suggestion min- ing system that participated in SemEval 2019 Task 9, SubTask A - Suggestion Mining from Online Reviews and Forums. Given some suggestions from online reviews and forums that can be classified into suggestion and non- suggestion classes. In this task, we combine the attention mechanism with the LSTM mod- el, which is the final system we submitted. The final submission achieves 14th place in Task 9, SubTask A with the accuracy of 0.6776. After the challenge, we train a series of neural net- work models such as convolutional neural net- work(CNN), TextCNN, long short-term mem- ory(LSTM) and C-LSTM. Finally, we make an ensemble on the predictions of these models and get a better result. 1 Introduction Suggestion mining can be defined as the extrac- tion of suggestions from unstructured text, where the term “suggestions” refers to the expressions of tips, advice, recommendations etc(Negi et al., 2019). These suggestions largely express positive and negative sentiments towards a given entity, but also tend to contain suggestions for improving the entity. Suggestion mining remains a relatively y- oung area compared to Sentiment Analysis, es- pecially in the context of recent advancements in neural network based approaches for learning fea- ture representations. In this task, suggestion min- ing that classified sentences into suggestion and non-suggestion classes was defined by the orga- nizer. In this paper, we mainly use an attention-based LSTM model(Hochreiter and Schmidhuber, 1997) for this task. The word-embedding used for al- l models in this task is Word2Vec. Then, the word vectors are fed into the long short-term memo- ry (LSTM) layer. Finally, an attention mechanis- Figure 1: An example from the SemEval 2019 Task 9 dataset m(Luong et al., 2015) is added into the neural net- works, and the prediction results are output via the softmax activation. What’s more, we try a num- ber of other models (such as the TextCNN(Kim, 2014), the C-LSTM(Zhou et al., 2015) and the attention-based Bi-LSTM(Lai et al., 2015) ) for comparative experiments. Furthermore we com- bine all of the above models to get results by soft voting. The rest of our paper is structured as follows. Section 2 introduces models. Section 3 describes data preparation. Experiments and evaluation are described in Section 4. The conclusions are drawn in Section 5. 2 Model For this task, we use 6 models for experiments. Among these models, the attention-based LSTM models can get the best results. This model com- bines the attention mechanism with the LSTM. The attention mechanism is a good solution to the information vanish problem in long sequence in- put situations. When …

Content truncated to 500 words