Sequential Learning of Convolutional Features for Effective Text Classification

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5658–5667, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5658 Sequential Learning of Convolutional Features for Effective Text Classification Avinash Madasu Samsung R&D Institute, Bangalore m.avinash@samsung.com Vijjini Anvesh Rao Samsung R&D Institute, Bangalore a.vijjini@samsung.com Abstract Text classification has been one of the ma- jor problems in natural language processing. With the advent of deep learning, convolu- tional neural network (CNN) has been a pop- ular solution to this task. However, CNNs which were first proposed for images, face many crucial challenges in the context of text processing, namely in their elementary blocks: convolution filters and max pool- ing. These challenges have largely been over- looked by the most existing CNN models pro- posed for text classification. In this paper, we present an experimental study on the funda- mental blocks of CNNs in text categorization. Based on this critique, we propose Sequen- tial Convolutional Attentive Recurrent Net- work (SCARN). The proposed SCARN model utilizes both the advantages of recurrent and convolutional structures efficiently in compar- ison to previously proposed recurrent convo- lutional models. We test our model on differ- ent text classification datasets across tasks like sentiment analysis and question classification. Extensive experiments establish that SCARN outperforms other recurrent convolutional ar- chitectures with significantly less parameters. Furthermore, SCARN achieves better perfor- mance compared to equally large various deep CNN and LSTM architectures. 1 Introduction Text classification is one of the major applications of Natural Language Processing (NLP). Text clas- sification involves classifying a text segment into different predefined categories. Sentiment analy- sis of product reviews, language detection, topic classification of various news articles are some of the problem statements of text classification. Prior to the success of deep learning, text clas- sification was dealt using lexicon based features. These primarily involve parts-of-speech (POS) tagging where verbs and adjectives are given more importance compared to other POS. Frequency based feature selection techniques like Bag-of- Words (BoW), Term frequency-Inverse document frequency (TF-IDF) have been used and the fea- tures are trained using machine learning classi- fiers like Logistic Regression (LR) or Naive Bayes (NB). These approaches provided strong baselines for text classification (Wang and Manning, 2012). However, sequential patterns and semantic struc- ture between words play a crucial role in decid- ing the category of a text. Traditional lexicon ap- proaches fail to capture such complexities. Since the success of deep learning, neural network ar- chitectures have outperformed traditional methods like BoW and count based feature selection tech- niques. Neural network architectures especially Convolutional Neural Networks (CNN) and Long Short Term Memory (Hochreiter and Schmidhu- ber, 1997) (LSTM) have achieved state-of-the-art results in text classification. Shallow CNN archi- tectures (Kim, 2014; Madasu and Rao, 2019a,b) achieved admirable results on text classification as well. Very deep CNN architectures were pro- posed based on character level features (Zhang et al., 2015) and word level features (Conneau et al., 2016) …

Content truncated to 500 words