Multinomial Adversarial Networks for Multi-Domain Text Classification

Proceedings of NAACL-HLT 2018, pages 1226–1240 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Multinomial Adversarial Networks for Multi-Domain Text Classification Xilun Chen Department of Computer Science Cornell Unversity Ithaca, NY, 14853, USA xlchen@cs.cornell.edu Claire Cardie Department of Computer Science Cornell Unversity Ithaca, NY, 14853, USA cardie@cs.cornell.edu Abstract Many text classification tasks are known to be highly domain-dependent. Unfortunately, the availability of training data can vary drasti- cally across domains. Worse still, for some domains there may not be any annotated data at all. In this work, we propose a multino- mial adversarial network1 (MAN) to tackle this real-world problem of multi-domain text clas- sification (MDTC) in which labeled data may exist for multiple domains, but in insufficient amounts to train effective classifiers for one or more of the domains. We provide theo- retical justifications for the MAN framework, proving that different instances of MANs are essentially minimizers of various f-divergence metrics (Ali and Silvey, 1966) among multi- ple probability distributions. MANs are thus a theoretically sound generalization of tradi- tional adversarial networks that discriminate over two distributions. More specifically, for the MDTC task, MAN learns features that are invariant across multiple domains by resort- ing to its ability to reduce the divergence among the feature distributions of each do- main. We present experimental results show- ing that MANs significantly outperform the prior art on the MDTC task. We also show that MANs achieve state-of-the-art performance for domains with no labeled data. 1 Introduction Text classification is one of the most fundamen- tal tasks in Natural Language Processing, and has found its way into a wide spectrum of NLP ap- plications, ranging from email spam detection and social media analytics to sentiment analysis and data mining. Over the past couple of decades, supervised statistical learning methods have be- come the dominant approach for text classification 1The source code of MAN is available at https:// github.com/ccsasuke/man. (e.g. McCallum et al. (1998); Kim (2014); Iyyer et al. (2015)). Unfortunately, many text classifica- tion tasks are highly domain-dependent in that a text classifier trained using labeled data from one domain is likely to perform poorly on another. In the task of sentiment classification, for example, the phrase “runs fast” is usually associated with positive sentiment in the sports domain; not so when a user is reviewing the battery of an elec- tronic device. In real applications, therefore, an adequate amount of training data from each do- main of interest is typically required, and this is expensive to obtain. Two major lines of work attempt to tackle this challenge: domain adaptation (Blitzer et al., 2007) and multi-domain text classification (MDTC) (Li and Zong, 2008). In domain adap- tation, the assumption is that there is some do- main with abundant training data (the source do- main), and the goal is to utilize knowledge learned from the source domain to help perform classifica- tions on another lower-resourced target domain.2 The focus of this work, MDTC, instead simulates an arguably more realistic …

Content truncated to 500 words