NL-FIIT at SemEval-2019 Task 9: Neural Model Ensemble for Suggestion Mining

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1218–1223 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1218 NL-FIIT at SemEval-2019 Task 9: Neural Model Ensemble for Suggestion Mining Samuel Pecar, Marian Simko, Maria Bielikova Slovak University of Technology in Bratislava Faculty of Informatics and Information Technologies Ilkovicova 2, 842 16 Bratislava, Slovakia {samuel.pecar, marian.simko, maria.bielikova}@stuba.sk Abstract In this paper, we present neural model archi- tecture submitted to the SemEval-2019 Task 9 competition: "Suggestion Mining from Online Reviews and Forums". We participated in both subtasks for domain specific and also cross- domain suggestion mining. We proposed a re- current neural network architecture that em- ploys Bi-LSTM layers and also self-attention mechanism. Our architecture tries to encode words via word representations using ELMo and ensembles multiple models to achieve bet- ter results. We performed experiments with different setups of our proposed model involv- ing weighting of prediction classes for loss function. Our best model achieved in official test evaluation score of 0.6816 for subtask A and 0.6850 for subtask B. In official results, we achieved 12th and 10th place in subtasks A and B, respectively. 1 Introduction Review-based portals and online forums contain plethora of user-generated text. We can con- sider customer reviews and inputs from online fo- rums as an important source of novel information. These texts often contain many different opinions, which are the subject of research in area of opinion mining. On the other hand, there can be also different types of information within these texts, such as suggestions. Unlike opinions, suggestions can ap- pear in different parts of text and also appear more sparsely. Suggestion mining, as defined in this task, can be realized as standard text classification. We perform classification to two classes, which are suggestion and non-suggestion. As presented by organizers, suggestion mining has different challenges (Negi et al., 2019): • Class imbalance - suggestions appear very sparsely in reviews and forums and most of the samples are negatively sampled, • Figurative expressions - expression can be of- ten found in social networks but it is not al- ways in form of suggestion, • Context dependency - some sentences can be viewed as a suggestion, if it appears in spe- cific domain or surrounded by specific sen- tences, • Long and complex sentences - suggestions can be expressed as only small part of orig- inal sentence, which can be much longer. Unlike opinions, suggestions can be more likely extracted also by pattern matching. We can ex- tract suggestions by different heuristic features and keywords, such as suggest, recommend, ad- vise (Negi and Buitelaar, 2015). Some works deal with domain terminology, thesaurus, linguis- tic parser and extraction rules (Brun and Hagege, 2013). Linguistic rules were also used for iden- tification and extraction in sentiment expression (Viswanathan et al., 2011). We believe that different extracted information from customer reviews and online forums can of- fer a valuable input for both customers and owners of products or forums and this information can …

Content truncated to 500 words