SINAI-DL at SemEval-2019 Task 5: Recurrent networks and data augmentation by paraphrasing

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 480–483 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 480 SINAI-DL at SemEval-2019 Task 5: Recurrent networks and data augmentation by paraphrasing Arturo Montejo-Ráez, Salud María Jiménez-Zafra, Miguel Ángel García-Cumbreras, Manuel Carlos Díaz-Galiano CEATIC / Universidad de Jaén Las Lagunillas s/n 23071 Jaén (Spain) {amontejo, sjzafra, magc, mcdiaz}@ujaen.es Abstract This paper describes the participation of the SINAI-DL team at Task 5 in SemEval 2019, called HatEval. We have applied some classic neural network layers, like word embeddings and LSTM, to build a neural classifier for both proposed tasks. Due to the small amount of training data provided compared to what is ex- pected for an adequate learning stage in deep architectures, we explore the use of paraphra- sing tools as source for data augmentation. Our results show that this method is promising, as some improvement has been found over non- augmented training sets. 1 Introduction We have participated in SemEval 2019 Task 5, na- med HatEval (Basile et al., 2019), which encou- rage participants to identify hate speech in tweets. The small amount of training data provided makes difficult to train a deep architecture, so strategies like transfer learning and data augmentation are explored in our work. A trained model for word embeddings in the two languages targeted by the tasks have been considered as transfer learning ap- proach. Paraphrasing the tweets has also been tes- ted for data augmentation, doubling the number of tweets available for training the network. Our results are promising for English, but no improvements have been found for Spanish. Furt- her analysis on the results and the quality of the paraphrasing tools used is needed, but the scores obtained in English encourage us to consider pa- raphrasing as a promising help in deep learning for natural language processing. The paper is organized as follows: Section 2 introduces the two main strategies used to train the neural network: data augmentation and trans- fer learning. In Section 3, task data is analyzed in order to define hyperparameters values. Section 4 describes the neural network architecture applied. Section 6 gives more details on the paraphrasing approach used to generate more training data. Ex- periments and results are given in Section 7. Fi- nally, Section 8 closes the contribution with some conclusions and proposals for future work. 2 Data augmentation and transfer learning Nowadays, deep neural architectures are popu- lating the scientific playground in many scena- rios: image recognition speech recognition (Gra- ves et al., 2013) and synthesis (Ze et al., 2013), and, of course, text classification (Zhang et al., 2015). But these supervised learning algorithms demands for valid use different requirements that sometimes are difficult to meet. One of the most difficult to overcame in some cases is the need for a large and varied learning dataset. When there is lack of data, two main strategies can be followed: transfer learning and data augmentation. Transfer learning. This approach proposes to train the network on …

Content truncated to 500 words