Intra-Topic Variability Normalization based on Linear Projection for Topic Classification

Proceedings of NAACL-HLT 2016, pages 441–446, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Intra-Topic Variability Normalization based on Linear Projection for Topic Classification Quan Liu†, Wu Guo†, Zhen-Hua Ling†, Hui Jiang‡, Yu Hu†§ † National Engineering Laboratory for Speech and Language Information Processing University of Science and Technology of China, Hefei, Anhui, China ‡ Department of Electrical Engineering and Computer Science York University, 4700 Keele Street, Toronto, Ontario, M3J 1P3, Canada § iFLYTEK Research, Hefei, China emails: quanliu@mail.ustc.edu.cn, guowu@ustc.edu.cn, zhling@ustc.edu.cn hj@cse.yorku.ca, yuhu@iflytek.com Abstract This paper proposes a variability normaliza- tion algorithm to reduce the variability be- tween intra-topic documents for topic classi- fication. Firstly, an optimization problem is constructed based on linear variability remov- able assumption. Secondly, a new feature s- pace for document representation is found by solving the optimization problem with kernel principle component analysis (KPCA). Final- ly, effective feature transformation is taken through linear projection. As for experiments, state-of-the-art SVM and KNN algorithm are adopted for topic classification respectively. Experimental results on a free-style conversa- tional corpus show that the proposed variabili- ty normalization algorithm for topic classifica- tion achieves 3.8% absolute improvement for micro-F1 measure. 1 Introduction Topic classification is now faced with the problem of enormous variability between documents due to the exponential growth of free-style unstructured texts in recent years. This paper treats variability as d- ifferences between text documents and aims at re- ducing the intra-topic document variability for bet- ter topic classification. There are various factors to cause the intra-topic variability problem, such as the different language usages of different persons (Chambers, 1995; Fillmore et al., 2014). In free- style conversations experimented in this paper, d- ifferent people would use very different words to express their opinions. Therefore, documents in a same topic could be quite different because of the intra-topic variability problem. In this work, we are interested in finding a ro- bust document representation strategy to address the intra-topic variability problem. Traditional method represents document by a high-dimensional TF-IDF vector based on the bag-of-word approach (Salton and McGill, 1986; Salton and Buckley, 1988). How- ever, the TF-IDF feature reveals little semantic sim- ilarity information between terms, which would increase the differences between intra-topic docu- ments when different words are used. Beyond the TF-IDF strategy, there are two class of techniques, i.e., unsupervised technique and supervised tech- nique for document representations. The unsuper- vised technique includes some latent semantic anal- ysis methods. The typical method is Latent Se- mantic Indexing (LSI) while the features estimated by LSI are linear combinations of the original fea- tures (Deerwester et al., 1990; Wang et al., 2013). Meanwhile, the popular Latent Dirichlet Allocation (Blei et al., 2003; Morchid et al., 2014) algorithm was proposed to represent document by a generative probabilistic model (Blei et al., 2003). Moreover, in recent years, many neural network based meth- ods have been investigated for document represen- tations (Hinton and Salakhutdinov, 2006; Srivasta- va et al., 2013; Le and Mikolov, 2014). For exam- ple, in (Le …

Content truncated to 500 words