Collective Document Classification with Implicit Inter-document Semantic Relationships

Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (*SEM 2015), pages 106–116, Denver, Colorado, June 4–5, 2015. Collective Document Classification with Implicit Inter-document Semantic Relationships Clinton Burford, Steven Bird and Timothy Baldwin Department of Computing and Information Systems The University of Melbourne, VIC 3010, Australia clint@burford.co sbird@unimelb.edu.au tb@ldwin.net Abstract This paper addresses the question of how doc- ument classifiers can exploit implicit infor- mation about document similarity to improve document classifier accuracy. We infer doc- ument similarity using simple n-gram over- lap, and demonstrate that this improves over- all document classification performance over two datasets. As part of this, we find that collective classification based on simple itera- tive classifiers outperforms the more complex and computationally-intensive dual classifier approach. 1 Introduction In machine learning, there is a rich tradition of re- search into the two tasks of: (1) “point-wise” clas- sification, where each instance is represented as an independent instance, and the predictive model at- tempts to learn a decision boundary to capture in- stances of a given class; and (2) graphical learn- ing and inference, where instances are connected in a graph, and learning/inference take place relative to the graph structure connecting those instances, based primarily on either conditional dependence (i.e. one event is dependent on the outcome of an- other) or “homophily” (i.e. the tendency for con- nected instances to share various properties).1 Var- ious joint models that combine the two have also been proposed, although in natural language pro- cessing at least, these have focused largely on con- ditional dependence, in the form of models such as 1In some tasks, it can also indicate heterophily, i.e. the ten- dency for connected instances to have contrasting properties, as we shall see for one of our two dataset. hidden Markov models (Rabiner and Juang, 1986) and conditional random fields (Lafferty et al., 2001), where independent properties of words, e.g., are combined with conditional dependencies based on their context of use to jointly predict the senses of all words in a given sentence (Ciaramita and John- son, 2003; Johannsen et al., 2014). This paper explores the utility of homophily within joint models for document-level semantic classification, focusing specifically on tasks which are not associated with any explicit graph structure. That is, we examine whether implicit semantic doc- ument links can improve the results of a point-wise (content-based) classification approach. Explicit inter-document links have been variously shown to improve document classifier performance, based on information sources including hyperlinks in web documents (Slattery and Craven, 1998; Oh et al., 2000; Yang et al., 2002), direct name-references in congressional debates (Thomas et al., 2006; Bur- foot et al., 2011; Stoyanov and Eisner, 2012), ci- tations in scientific papers (Giles et al., 1998; Lu and Getoor, 2003; McDowell et al., 2007), and user mentions or retweets in social media (Jiang et al., 2011; Tan et al., 2011). However, document col- lections often don’t contain explicit inter-document links, limiting the practical usefulness of such meth- ods. In this paper, we seek to …

Content truncated to 500 words