Automatic Title Generation for Spoken Broadcast News

Automatic Title Generation for Spoken Broadcast News Rong Jin Language Technology Institute Carnegie Mellon University Pittsburgh, PA 15213 412-268-7003 rong+@cs.cmu.edu Alexander G. Hauptmann School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 412-268-1448 alex+@cs.cmu.edu ABSTRACT In this paper, we implemented a set of title generation methods using training set of 21190 news stories and evaluated them on an independent test corpus of 1006 broadcast news documents, comparing the results over manual transcription to the results over automatically recognized speech. We use both F1 and the average number of correct title words in the correct order as metric. Overall, the results show that title generation for speech recognized news documents is possible at a level approaching the accuracy of titles generated for perfect text transcriptions. Keywords Machine learning, title generation 1. INTRODUCTION To create a title for a document is a complex task. To generate a title for a spoken document becomes even more challenging because we have to deal with word errors generated by speech recognition. Historically, the title generation task is strongly connected to traditional summarization because it can be thought of extremely short summarization. Traditional summarization has emphasized the extractive approach, using selected sentences or paragraphs from the document to provide a summary. The weaknesses of this approach are inability of taking advantage of the training corpus and producing summarization with small ratio. Thus, it will not be suitable for title generation tasks. More recently, some researchers have moved toward “learning approaches” that take advantage of training data. Witbrock and Mittal [1] have used Naïve Bayesian approach for learning the document word and title word correlation. However they limited their statistics to the case that the document word and the title word are same surface string. Hauptmann and Jin [2] extended this approach by relaxing the restriction. Treating title generation problem as a variant of Machine translation problem, Kennedy and Hauptmann [3] tried the iterative Expectation-Maximization algorithm. To avoid struggling with organizing selected title words into human readable sentence, Hauptmann [2] used K nearest neighbour method for generating titles. In this paper, we put all those methods together and compare their performance over 1000 speech recognition documents. We decompose the title generation problem into two parts: learning and analysis from the training corpus and generating a sequence of title words to form the title. For learning and analysis of training corpus, we present five different learning methods for comparison: Naïve Bayesian approach with limited vocabulary, Naïve Bayesian approach with full vocabulary, K nearest neighbors, Iterative Expectation- Maximization approach, Term frequency and inverse document frequency method. More details of each approach will be presented in Section 2. For the generating part, we decompose the issues involved as follows: choosing appropriate title words, deciding how many title words are appropriate for this document title, and finding the correct sequence of title words that forms a readable title ‘sentence’. The outline of this paper is as follows: Section 1 gave an introduction to the title generation problem. The details of …

Content truncated to 500 words