A New Probabilistic Model for Title Generation

A New Probabilistic Model for Title Generation Rong Jin Language Technology Institute Carnegie Mellon University 5000 Forbes Ave. Pittsburgh, PA15213, U. S. A. rong+@cs.cmu.edu Alexander G. Hauptmann Department of Computer Science Carnegie Mellon University 5000 Forbes Ave. Pittsburgh, PA15213, U. S. A. alex+@cs.cmu.edu Abstract Title generation is a complex task involving both natural language understanding and natural language synthesis. In this paper, we propose a new probabilistic model for title generation. Different from the previous statistical models for title generation, which treat title generation as a generation process that converts the ‘document representation’ of information directly into a ‘title representation’ of the same information, this model introduces a hidden state called ‘information source’ and divides title generation into two steps, namely the step of distilling the ‘information source’ from the observation of a document and the step of generating a title from the estimated ‘information source’. In our experiment, the new probabilistic model outperforms the previous model for title generation in terms of both automatic evaluations and human judgments. Introduction Compared with a document, a title provides a compact representation of the information and therefore helps people quickly capture the main idea of a document without spending time on the details. Automatic title generation is a complex task, which not only requires finding the title words that reflects the document content but also demands ordering the selected title words into human readable sequence. Therefore, it involves in both nature language understanding and nature language synthesis, which distinguishes title generation from other seemingly similar tasks such as key phrase extraction or automatic text summarization where the main concern of tasks is identify important information units from documents (Mani & Maybury., 1999). The statistical approach toward title generation has been proposed and studied in the recent publications (Witbrock & Mittal, 1999; Kennedy & Hauptmann, 2000; Jin & Hauptmann, 2001). The basic idea is to first learn the correlation between the words in titles (title words) and the words in the corresponding documents (document words) from a given training corpus consisting of document-title pairs, and then apply the learned title-word-document-word correlations to generate titles for unseen documents. Witbrock and Mittal (1999) proposed a statistical framework for title generation where the task of title generation is decomposed into two phases, namely the title word selection phase and the title word ordering phase. In the phase of title word selection, each title word is scored based on its indication of the document content. During the title word ordering phase, the ‘appropriateness’ of the word order in a title is scored using ngram statistical language model. The sequence of title words with highest score in both title word selection phase and title word ordering phase is chosen as the title for the document. The follow-ups within this framework mainly focus on applying different approaches to the title word selection phase (Jin & Hauptmann, 2001; Kennedy & Hauptmann, 2000). However, there are two problems with this framework for title generation. They are: • A problem with the …

Content truncated to 500 words