Solving Feature Sparseness in Text Classification using Core-Periphery Decomposition

Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (*SEM), pages 255–264 New Orleans, June 5-6, 2018. c⃝2018 Association for Computational Linguistics Solving Feature Sparseness in Text Classification using Core-Periphery Decomposition Xia Cui, Sadamori Kojaku, Naoki Masuda, and Danushka Bollegala Department of Computer Science, University of Liverpool Department of Engineering Mathematics, University of Bristol {xia.cui, danushka.bollegala}@liverpool.ac.uk {sadamori.koujaku, naoki.masuda}@bristol.ac.uk Abstract Feature sparseness is a problem common to cross-domain and short-text classification tasks. To overcome this feature sparseness problem, we propose a novel method based on graph decomposition to find candidate fea- tures for expanding feature vectors. Specif- ically, we first create a feature-relatedness graph, which is subsequently decomposed into core-periphery (CP) pairs and use the periph- eries as the expansion candidates of the cores. We expand both training and test instances using the computed related features and use them to train a text classifier. We observe that prioritising features that are common to both training and test instances as cores dur- ing the CP decomposition to further improve the accuracy of text classification. We evalu- ate the proposed CP-decomposition-based fea- ture expansion method on benchmark datasets for cross-domain sentiment classification and short-text classification. Our experimental re- sults show that the proposed method consis- tently outperforms all baselines on short-text classification tasks, and perform competitively with pivot-based cross-domain sentiment clas- sification methods. 1 Introduction Short-texts are abundant on the Web and ap- pear in various different formats such as micro- blogs (Kwak et al., 2010), Question and Answer (QA) forums, review sites, Short Message Ser- vice (SMS), email, and chat messages (Cong et al., 2008; Thelwall et al., 2010). Unlike lengthy re- sponses that take time to both compose and to read, short responses have gained popularity par- ticularly in social media contexts. Considering the steady growth of mobile devices that are physi- cally restricted to compact keyboards, which are suboptimal for entering lengthy text inputs, it is safe to predict that the amount of short-texts will continue to grow in the future. Considering the importance and the quantity of the short-texts in various web-related tasks, such as text classifica- tion (kun Wang et al., 2012; dos Santos and Gatti, 2014), and event prediction (Sakaki et al., 2010), it is important to be able to accurately represent and classify short-texts. Compared to performing text mining on longer texts (Guan et al., 2009; Su et al., 2011; Yogatama and Smith, 2014), for which dense and diverse fea- ture representations can be created relatively eas- ily, handling of shorter texts poses several chal- lenges. The number of features that are present in a given short-text will be a small fraction of the set of all features that exist in all of the train instances. Moreover, frequency of a feature in a short-text will be small, which makes it difficult to reliably estimate the salience of a feature using term frequency-based methods. This is known as the feature sparseness problem in text classifica- tion. Feature sparseness is not unique to short- text classification …

Content truncated to 500 words