Adaptive Convolution for Text Classification

Proceedings of NAACL-HLT 2019, pages 2475–2485 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2475 Adaptive Convolution for Text Classification Byung-Ju Choi Jun-Hyung Park SangKeun Lee Department of Computer Science and Engineering Korea University, Korea {bj1123, irish07, yalphy}@korea.ac.kr Abstract In this paper, we present an adaptive convolution for text classification to give stronger flexibility to convolutional neural networks (CNNs). Unlike traditional convolutions that use the same set of filters regardless of different inputs, the adaptive convolution employs adaptively generated convolutional filters that are conditioned on inputs. We achieve this by attaching filter-generating networks, which are carefully designed to generate input-specific filters, to convolution blocks in existing CNNs. We show the efficacy of our approach in existing CNNs based on our performance evaluation. Our evaluation indicates that adaptive convolutions improve all the baselines, without any exception, as much as up to 2.6 percentage point in seven benchmark text classification datasets. 1 Introduction Text classification assigns topics to texts by understanding the semantics of the texts. It is one of the fundamental tasks in natural language processing (NLP) which has a broad range of applications, including web search (Broder et al., 2007), contextual advertising (Lee et al., 2013), and user profiling (Kazai et al., 2016). Traditional approaches to text classification use sparse representations of text, such as bag-of-words (Lodhi et al., 2002). To date, neural network- based text embedding techniques, particularly convolutional neural networks (CNNs) (Kim, 2014; Zhang et al., 2015; Wang et al., 2018b) have shown remarkable results in text classification. One of the driving forces of CNNs is a convolution operation. It screens local information which appear in inputs (either input texts or outputs from the previous convolution block) by convolving a set of filters with inputs. In the context of text classification, this operation is analogous to questions and answers. Convolutional filters are like questions asking for the intensity of particular patterns in receptive fields. Outputs of convolution operations are the answers from the inputs to the questions. CNNs derive the right class with stacked convolution blocks1. On this point, CNNs can be likened to players of the twenty questions who guess the answers by iteratively asking questions and receiving information. However, differences exist between humans and traditional CNNs in the manner in which they play this game. Humans adaptively ask questions by fully utilizing information they have obtained. If players have narrowed the answer down to the name of a person, they would not want to ask questions such as, “Does that have four legs?”. Rather, they would prefer questions related to the target’s profession or origin that are practical for inferring the answer. In contrast, typical CNNs use the same set of filters in any circumstances (Kim, 2014; Johnson and Zhang, 2017; Wang et al., 2018b). This may hamper CNNs from leveraging the information they have as intermediate hidden representations of inputs processed in consecutive convolution operations, and focusing their capacity on disentangling uncertainty. Motivated by this, we propose an adaptive convolution …

Content truncated to 500 words