DS at SemEval-2019 Task 9: From Suggestion Mining with neural networks to adversarial cross-domain classification

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1192–1198 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1192 DS at SemEval-2019 Task 9: From Suggestion Mining with neural networks to adversarial cross-domain classification Tobias Cabanski t.cabanski@posteo.de Abstract Suggestion Mining is the task of classifying sentences into suggestions or non-suggestions. SemEval-2019 Task 9 sets the task to mine suggestions from online texts. For each of the two subtasks, the classification has to be ap- plied on a different domain. Subtask A ad- dresses the domain of posts in suggestion on- line forums and comes with a set of training examples, that is used for supervised training. A combination of LSTM and CNN networks is constructed to create a model which uses BERT word embeddings as input features. For subtask B, the domain of hotel reviews is re- garded. In contrast to subtask A, no labeled data for supervised training is provided, so that additional unlabeled data is taken to apply a cross-domain classification. This is done by using adversarial training of the three model parts label classifier, domain classifier and the shared feature representation. For subtask A, the developed model archives a F1-score of 0.7273, which is in the top ten of the leader board. The F1-score for subtask B is 0.8187 and is ranked in the top five of the submissions for that task. 1 Introduction For getting feedback from costumers or users, an organization often uses forums and social media channels. Also ratings of products on rating plat- forms can be an useful feedback to make a product better. The feedback from a customer can be in the form of a suggestion which appears in a rating text or is directly asked from the customer. The task of suggestion mining can be defined as the extrac- tion of sentences that contain suggestions from un- structured text (Negi et al., 2018). SemEval-2019 Task 9 Subtask A provides the challenge to do sug- gestion mining on data from an online suggestion forum. For that subtask, a train and validation set is provided so that it is possible to apply super- vised training. For subtask B, suggestions in hotel reviews should be identified. An additional dif- ficulty for that subtask is that no labeled data is given except a small validation set, which is not al- lowed to be used for supervised training. For both tasks, silver standard datasets are allowed to use, which means that data that is likely to belong to a certain class can be taken as long as it is not manu- ally labeled. A more detailed task description can be found in (Negi et al., 2019). 2 Data The dataset for subtask A provides an overall count of 8500 examples, where 6415 examples are labeled as non-suggestion and 2085 as suggestion. Also a trial dataset is provided that contains 592 examples, divided in 296 examples for each class. Every example contains only one sentence, which could be part of a whole …

Content truncated to 500 words