Building Semantic Perceptron Net for Topic Spotting

Building Semantic Perceptron Net for Topic Spotting Jimin Liu and Tat-Seng Chua School of Computing National University of Singapore SINGAPORE 117543 {liujm, chuats}@comp.nus.edu.sg Abstract This paper presents an approach to automatically build a semantic perceptron net (SPN) for topic spotting. It uses context at the lower layer to select the exact meaning of key words, and employs a combination of context, co-occurrence statistics and thesaurus to group the distributed but semantically related words within a topic to form basic semantic nodes. The semantic nodes are then used to infer the topic within an input document. Experiments on Reuters 21578 data set demonstrate that SPN is able to capture the semantics of topics, and it performs well on topic spotting task. 1. Introduction Topic spotting is the problem of identifying the presence of a predefined topic in a text document. More formally, given a set of n topics together with a collection of documents, the task is to determine for each document the probability that one or more topics is present in the document. Topic spotting may be used to automatically assign subject codes to newswire stories, filter electronic emails and on- line news, and pre-screen document in information retrieval and information extraction applications. Topic spotting, and its related problem of text categorization, has been a hot area of research for over a decade. A large number of techniques have been proposed to tackle the problem, including: regression model, nearest neighbor classification, Bayesian probabilistic model, decision tree, inductive rule learning, neural network, on-line learning, and, support vector machine (Yang & Liu, 1999; Tzeras & Hartmann, 1993). Most of these methods are word-based and consider only the relationships between the features and topics, but not the relationships among features. It is well known that the performance of the word-based methods is greatly affected by the lack of linguistic understanding, and, in particular, the inability to handle synonymy and polysemy. A number of simple linguistic techniques has been developed to alleviate such problems, ranging from the use of stemming, lexical chain and thesaurus (Jing & Tzoukermann, 1999; Green, 1999), to word-sense disambiguation (Chen & Chang, 1998; Leacock et al, 1998; Ide & Veronis, 1998) and context (Cohen & Singer, 1999; Jing & Tzoukermann, 1999). The connectionist approach has been widely used to extract knowledge in a wide range of information processing tasks including natural language processing, information retrieval and image understanding (Anderson, 1983; Lee & Dubin, 1999; Sarkas & Boyer, 1995; Wang & Terman, 1995). Because the connectionist approach closely resembling human cognition process in text processing, it seems natural to adopt this approach, in conjunction with linguistic analysis, to perform topic spotting. However, there have been few attempts in this direction. This is mainly because of difficulties in automatically constructing the semantic networks for the topics. In this paper, we propose an approach to automatically build a semantic perceptron net (SPN) for topic spotting. The SPN is a connectionist model with hierarchical structure. It uses a combination of context, co-occurrence statistics and …

Content truncated to 500 words