One-class Text Classification with Multi-modal Deep Support Vector Data Description

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3378–3390 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3378 One-class Text Classification with Multi-modal Deep Support Vector Data Description Chenlong Hu1, Yukun Feng1, Hidetaka Kamigaito1, Hiroya Takamura1,2 and Manabu Okumura1 1Tokyo Institute of Technology 2National Institute of Advanced Industrial Science and Technology (AIST) {huchenlong,yukun,kamigaito,oku}@lr.pi.titech.ac.jp takamura@pi.titech.ac.jp Abstract This work presents multi-modal deep SVDD (mSVDD) for one-class text classification. By extending the uni-modal SVDD to a multi- ple modal one, we build mSVDD with mul- tiple hyperspheres, that enable us to build a much better description for target one-class data. Additionally, the end-to-end architecture of mSVDD can jointly handle neural feature learning and one-class text learning. We also introduce a mechanism for incorporating nega- tive supervision in the absence of real negative data, which can be beneficial to the mSVDD model. We conduct experiments on Reuters and 20 Newsgroup datasets, and the experi- mental results demonstrate that mSVDD out- performs uni-modal SVDD and mSVDD can get further improvements when negative super- vision is incorporated. 1 Introduction One-Class Classification (OCC), a special classi- fication problem, aims to learn a model on the ba- sis of training samples only from one class. The learned model is expected to make an accurate de- scription of the class (so called target or normal) and then to distinguish the target from samples for negative classes during testing (Moya et al., 1993; Tax, 2002). The one-class classification problem has arisen in many real-world applications, includ- ing anomaly or novelty detection (Roberts, 1999; Chandola et al., 2010; Gupta et al., 2013), bioin- formatics (Alashwal et al., 2006), and especially computer vision (Rodner et al., 2011; Ruff et al., 2018). One-class text classification would be benefi- cial to the scenario where anomalous text contents (e.g., web pages, spam emails) (Yu et al., 2004) need to be detected, and only a positive training corpus is available. One of the early work on one- class text classification is Manevitz and Yousef (2001), who implemented versions of one-class support vector machines (OC-SVM) (Sch¨olkopf et al., 2001) and showed good performances over the Reuters dataset (Dumais et al., 1998). OC- SVM and support vector data description (SVDD) (Tax and Duin, 2004) are boundary-based meth- ods (Tax, 2002). Both try to describe the target data using a boundary. SVDD learns an optimal hy- persphere with the minimum radius to include the most target data, while OC-SVM builds a hyper- plane to maximally separate the data points from the origin where outlier examples lie around. Reconstruction-based approaches, including Au- toEncoder (Jacobs, 1995) and principal component analysis (PCA) (Bishop, 1995), which aim to learn a more compact representation for the description of target data. The compact representation could be a set of prototypes or subspaces obtained by opti- mizing a reconstruction error on the target training data. Regarding the features for representing text in OCC, document-to-word co-occurrence matrices or hand-crafted features have been commonly used in most of …

Content truncated to 500 words