MIDAS at SemEval-2019 Task 9: Suggestion Mining from Online Reviews using ULMFit

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1213–1217 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1213 MIDAS at SemEval-2019 Task 9: Suggestion Mining from Online Reviews using ULMFiT Sarthak Anand,3 Debanjan Mahata,1 Kartik Aggarwal, 3 Laiba Mehnaz,2 Simra Shahid,2 Haimin Zhang,1 Yaman Kumar,5 Rajiv Ratn Shah,4 Karan Uppal1 1Bloomberg, USA, 2DTU-Delhi, India, 3NSIT-Delhi, India, 4IIIT-Delhi, India, 5Adobe, India, sarthaka.ic@nsit.net.in, dmahata@bloomberg.net, kartik.mp.16@nsit.net.in, laibamehnaz@dtu.ac.in, simrashahid bt2k16@dtu.ac.in, hzhang449@bloomberg.net, ykumar@adobe.com, rajivratn@iiitd.ac.in, kuppal8@bloomberg.net Abstract In this paper we present our approach and the system description for Sub Task A of SemEval 2019 Task 9: Suggestion Mining from On- line Reviews and Forums. Given a sentence, the task asks to predict whether the sentence consists of a suggestion or not. Our model is based on Universal Language Model Fine- tuning for Text Classification. We apply vari- ous pre-processing techniques before training the language and the classification model. We further provide detailed analysis of the results obtained using the trained model. Our team ranked 10th out of 34 participants, achieving an F1 score of 0.7011. We publicly share our implementation1. 1 Introduction and Background Suggestion mining can be defined as the process of identifying and extracting sentences from un- structured text that contain suggestion (Negi et al., 2018). Suggestions in the form of unstructured text could be found in various social media plat- forms, discussion forums, review websites and blogs. They are often expressed in the form of ad- vice, tips, recommendations, warnings, things to do, and various other forms in an explicit as well as an implicit way. Identifying and retrieving suggestions from text can be useful in an industrial setting for enhanc- ing a product, summarizing opinions of the con- sumers, giving recommendations and as an aid in decision making process (Jijkoun et al., 2010). For normal users of online platforms it could help in seeking advice related to general topics of inter- est like travel, health, food, shopping, education, 1https://github.com/isarth/SemEval9_ MIDAS and many more. Given the abundance of textual information in the Internet about a variety of top- ics, suggestion mining is certainly an useful task interesting to researchers working in academia as well as industry. Most of the previous efforts in the direction of understanding online opinions and reactions have been limited to developing methods for areas like sentiment analysis and opinion mining (Medhat et al., 2014; Baghel et al., 2018; Kapoor et al., 2018; Mahata et al., 2018a,b; Jangid et al., 2018; Meghawat et al., 2018; Shah and Zimmermann, 2017). Mining and understanding suggestions can open new areas to study consumer behavior and tapping nuggets of information that could be di- rectly linked with the development and enhance- ment of products (Brun and Hagege, 2013; Dong et al., 2013; Ramanand et al., 2010), improve cus- tomer experiences (Negi and Buitelaar, 2015), and aid in understanding the linguistic nuances of giv- ing advice (Wicaksono and Myaeng, 2013). Suggestion mining is a relatively new domain and is challenged by problems such as ambiguity in task formulation …

Content truncated to 500 words