DBMS-KU at SemEval-2019 Task 9: Exploring Machine Learning Approaches in Classifying Text as Suggestion or Non-Suggestion

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1185–1191 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1185 DBMS-KU at SemEval-2019 Task 9: Exploring Machine Learning Approaches in Classifying Text as Suggestion or Non-Suggestion Tirana Noor Fatyanosa1, Al Hafiz Akbar Maulana Siagian1,3, Masayoshi Aritsugi2 1Computer Science and Electrical Engineering Graduate School of Science and Technology, Kumamoto University, Japan 2Big Data Science and Technology Faculty of Advanced Science and Technology, Kumamoto University, Japan 3Indonesian Institute of Sciences, Indonesia {fatyanosa,alha002}@dbms.cs.kumamoto-u.ac.jp aritsugi@cs.kumamoto-u.ac.jp Abstract This paper describes the participation of DBMS-KU team in the SemEval 2019 Task 9, that is, suggestion mining from online reviews and forums. To deal with this task, we explore several machine learning ap- proaches, i.e., Random Forest (RF), Logis- tic Regression (LR), Multinomial Naive Bayes (MNB), Linear Support Vector Classification (LSVC), Sublinear Support Vector Classifica- tion (SSVC), Convolutional Neural Network (CNN), and Variable Length Chromosome Ge- netic Algorithm-Naive Bayes (VLCGA-NB). Our system obtains reasonable results of F1- Score 0.47 and 0.37 on the evaluation data in Subtask A and Subtask B, respectively. In particular, our obtained results outperform the baseline in Subtask A. Interestingly, the results seem to show that our system could perform well in classifying Non-suggestion class. 1 Introduction Nowadays, a huge number of texts are posted in online reviews or discussion forums. Such media can be a valuable source for obtaining a suggestion about products or services (Negi and Buitelaar, 2015; Negi et al., 2016). The obtained suggestion is not only useful for readers but also important information for stakeholders (Negi et al., 2016). Indeed, such advice can be used to improving the quality of products or giving helpful recommenda- tions (Brun and Hagege, 2013). However, identi- fying a suggestion from a lot of reviews or com- ments needs extra effort and time. Moreover, such online texts are mostly in unstructured form (Negi et al., 2018; Negi and Buitelaar, 2017). Thus, au- tomatically mining the suggestion from given texts is challenging and significant (Negi et al., 2016). Suggestion mining is relatively a new research interest in text classification tasks (Negi and Buite- laar, 2015). Several studies have initiated to min- ing suggestions from online texts (Negi et al., 2018; Negi and Buitelaar, 2017; Negi et al., 2016; Negi, 2016; Negi and Buitelaar, 2015; Brun and Hagege, 2013; Ramanand et al., 2010; Dong et al., 2013; Wicaksono and Myaeng, 2012, 2013). Particularly, (Negi and Buitelaar, 2015; Brun and Hagege, 2013; Ramanand et al., 2010) have tried to identify suggestions from customer reviews. Meanwhile, (Negi, 2016; Dong et al., 2013; Wicaksono and Myaeng, 2012, 2013) have mined such advice by using Twitter or discussion forums dataset. Then, (Negi et al., 2018; Negi and Buitelaar, 2017) have utilized WikiHow and open domain corpora for their work. However, they concluded that it is not easy to identify sugges- tion texts automatically. In other words, it still has room to improving the classification result in the suggestion mining task. The task of suggestion mining from …

Content truncated to 500 words