PLSI Utilization for Automatic Thesaurus Construction

PLSI Utilization for Automatic Thesaurus Construction Masato Hagiwara, Yasuhiro Ogawa, and Katsuhiko Toyama Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya, JAPAN 464-8603 {hagiwara, yasuhiro, toyama}@kl.i.is.nagoya-u.ac.jp Abstract. When acquiring synonyms from large corpora, it is important to deal not only with such surface information as the context of the words but also their latent semantics. This paper describes how to utilize a latent semantic model PLSI to acquire synonyms automatically from large corpora. PLSI has been shown to achieve a better performance than conventional methods such as tf·idf and LSI, making it applicable to automatic thesaurus construction. Also, various PLSI techniques have been shown to be effective including: (1) use of Skew Divergence as a distance/similarity measure; (2) removal of words with low frequencies, and (3) multiple executions of PLSI and integration of the results. 1 Introduction Thesauri, dictionaries in which words are arranged according to meaning, are one of the most useful linguistic sources, having a broad range of applications, such as information retrieval and natural language understanding. Various thesauri have been constructed so far, including WordNet [6] and Bunruigoihyo [14]. Conventional thesauri, however, have largely been compiled by groups of language experts, making the construction and maintenance cost very high. It is also difficult to build a domain-specific thesaurus flexibly. Thus it is necessary to construct thesauri automatically using computers. Many studies have been done for automatic thesaurus construction. In doing so, synonym acquisition is one of the most important techniques, although a thesaurus gen- erally includes other relationships than synonyms (e.g., hypernyms and hyponyms). To acquire synonyms automatically, contextual features of words, such as co-occurrence and modification are extracted from large corpora and often used. Hindle [7], for ex- ample, extracted verb-noun relationships of subjects/objects and their predicates from a corpus and proposed a method to calculate similarity of two words based on their mu- tual information. Although methods based on such raw co-occurrences are simple yet effective, in a naive implementation some problems arise: namely, noises and sparse- ness. Being a collection of raw linguistic data, a corpus generally contains meaningless information, i.e., noises. Also, co-occurrence data extracted from corpora are often very sparse, making them inappropriate for similarity calculation, which is also known as the “zero frequency problem.” Therefore, not only surface information but also latent se- mantics should be considered when acquiring synonyms from large corpora. Several latent semantic models have been proposed so far, mainly for information retrieval and document indexing. The most commonly used and prominent ones are La- tent Semantic Indexing (LSI) [5] and Probabilistic LSI (PLSI) [8]. LSI is a geometric R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 334–345, 2005. c⃝ Springer-Verlag Berlin Heidelberg 2005 PLSI Utilization for Automatic Thesaurus Construction 335 model based on the vector space model. It utilizes singular value decomposition of the co-occurrence matrix, an operation similar to principal component analysis, to auto- matically extract major components that contribute to the indexing of documents. It can alleviate the noise and sparseness …

Content truncated to 500 words