Topic Analysis Using a Finite Mixture Model

Topic Analysis Using a Finite Mixture Model Hang Li and Kenji Yamanishi NEC Corporation {lihang,yamanisi} @ccm.cl.nec.co.jp Abstract We address the issue of 'topic analysis,' by which is determined a text's topic structure, which indicates what topics are included in a text, and how topics change within the text. We propose a novel approach to this issue, one based on statistical modeling and learning. We represent topics by means of word clusters, and employ a finite mixture model to repre- sent a word distribution within a text. Our experimental results indicate that our method significantly outperforms a method that com- bines existing techniques. 1 Introduction -:We consider here the issue of 'topic analysis,' by which is determined a text's topic struc- ture, which indicates what topics are included in a text and how topics change within the text. Topic analysis consists of two main tasks: topic identification and text segmen- tation (based on topic changes). Topic analysis is extremely useful in a vari- ety of text processing applications. For exam- plea it can be used in the automatic indexing of texts for purposes of information retrieval. With it, one can understand what the main topics and subtopics of a text are, and where those subtopics lie within the text. To the best of our knowledge, however, no previous study has so far dealt with the topic analysis problem in the above sense. The most closely related are key word extraction and text segmentation. A keyword extrac- tion method (e.g., that using tf-idf (Salton and Yang, 1973)) generally extracts from a text key words which represent topics within the text, but it does not conduct segmenta- tion. A segmentation method (e.g., TextTil- ing (Hearst, 1997)) generally segments a text into blocks (paragraphs) in accord with topic changes within the text, but it does not iden- tify (or label) by itself the topics discussed in each of the blocks. The purpose of tMs paper is to provide a single framework for conducting topic analy- sis, i.e., performing both topic identification and text segmentation. The key characteristics of our framework are 1) representing a topic by means of a clus- ter of words that are closely related to the topic, and 2) employing a stochastic model, called a .finite mixture model (e.g., (Everitt and Hand, 1981)), to represent a word dis- tribution within a text. The finite mixture model has a hierarchical structure of probabil- ity distributions. The first level is a probabil- ity distribution of topics (topic distribution). The second level consists of probability distri- butions of words included within topics (word distributions). These word distributions are linearly combined to represent a word distri- bution within a text, with the topic distribu- tion being used as the coefficient vector. Here- after we refer to a finite mixture model hav- ing this structure as a stochastic topic model (STM). Before conducting topic analysis, we create word clusters (topics) on the basis of word co- occurrence in corpus data. We have devel- oped a new …

Content truncated to 500 words