Towards Integration of Statistical Hypothesis Tests into Deep Neural Networks

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5551–5557 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5551 Towards Integration of Statistical Hypothesis Tests into Deep Neural Networks Ahmad Aghaebrahimian Zurich University of Applied Sciences Switzerland agha@zhaw.ch Mark Cieliebak Zurich University of Applied Sciences Switzerland ciel@zhaw.ch Abstract We report our ongoing work about a new deep architecture working in tandem with a statis- tical test procedure for jointly training texts and their label descriptions for multi-label and multi-class classification tasks. A statistical hypothesis testing method is used to extract the most informative words for each given class. These words are used as a class description for more label-aware text classification. In- tuition is to help the model to concentrate on more informative words rather than more fre- quent ones. The model leverages the use of la- bel descriptions in addition to the input text to enhance text classification performance. Our method is entirely data-driven, has no depen- dency on other sources of information than the training data, and is adaptable to different clas- sification problems by providing appropriate training data without major hyper-parameter tuning. We trained and tested our system on several publicly available datasets, where we managed to improve the state-of-the-art on one set with a high margin, and to obtain competi- tive results on all other ones. 1 Introduction Text classification is a complex problem in Natural Language Processing (NLP) with lots of applica- tions from sentiment analysis (Liu, 2015) to ques- tion answering (Aghaebrahimian and Jurˇc´ıˇcek, 2016b,a; Yu et al., 2014) or abusive language de- tection (von Gr¨unigen et al., 2018; Founta et al., 2018), to name just a few. Text classification is de- fined as the task of assigning a certain pre-defined class to a document. The number of classes can be arbitrarily large in multi-class classification, whereas there are only two classes for binary classification. In multi- label classification, the number of labels attached to each document is not known and usually larger than one, while in multi-class classification, only one class is assigned to each document. There exist numerous approaches for text classi- fication, ranging from simple hand-crafted lexical- level features with Naive Bayes or Support Vec- tor Machines (SVM) (Wang and Manning, 2012) to self-learning approaches with Deep Neural Net- works (DNN) (Deriu and Cieliebak, 2017). For the latter, several architectures such as Con- volutional or Recurrent Neural Networks (CNN or RNN) (Shen et al., 2017; Wang et al., 2018b) have been proposed. These architectures learn differ- ent levels of textual representation in their layers, which are an essential source of information for the classification process. As an alternative, at- tention networks are also introduced (Bahdanau et al., 2015; Yang et al., 2016) to capture the fea- tures with the highest discriminative power regard- ing the class and irrespective of their distance. On the other hand, the field of Statistics has since long developed and optimized various methods to cap- ture ‘relevant’ properties …

Content truncated to 500 words