Bayesian Learning in Text Summarization

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 249–256, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Bayesian Learning in Text Summarization Tadashi Nomoto National Institute of Japanese Literature 1-16-10 Yutaka Shinagawa Tokyo 142-8585 Japan nomoto@acm.org Abstract The paper presents a Bayesian model for text summarization, which explicitly en- codes and exploits information on how hu- man judgments are distributed over the text. Comparison is made against non Bayesian summarizers, using test data from Japanese news texts. It is found that the Bayesian approach generally lever- ages performance of a summarizer, at times giving it a significant lead over non- Bayesian models. 1 Introduction Consider figure 1. What is shown there is the pro- portion of the times that sentences at particular lo- cations are judged as relevant to summarization, or worthy of inclusion in a summary. Each panel shows judgment results on 25 Japanese texts of a particular genre; columns (G1K3), editorials (G2K3) and news stories (G3K3). All the documents are from a sin- gle Japanese news paper, and judgments are elicited from some 100 undergraduate students. While more will be given on the details of the data later (Sec- tion 3.2), we can safely ignore them here. Each panel has the horizontal axis representing lo- cation or order of sentence in a document, and the vertical axis the proportion of the times sentences at particular locations are picked as relevant to summa- rization. Thus in G1K3, we see that the first sentence (to appear in a document) gets voted for about 12% of the time, while the 26th sentence is voted for less than 2% of the time. Curiously enough, each of the panels exhibits a distinct pattern in the way votes are spread across a document: G1K3 has the distribution of votes (DOV) with sharp peaks around 1 and 14; in G2K3, the distribution is peaked around 1, with a small bump around 19; in G3K3, the distribution is sharply skewed to the left, indicating that the majority of votes went to the initial section of a document. What is interesting about the DOV is that we could take it as indicating a collective preference for what to extract for a summary. A question is then, can we somehow exploit the DOV in summarization? To our knowledge, no prior work seems to exist that addresses the question. The paper discusses how we could do this under a Bayesian modeling frame- work, where we explicitly represent and make use of the DOV by way of Dirichlet posterior (Congdon, 2003).1 2 Bayesian Model of Summaries Since the business of extractive summarization, such as one we are concerned with here, is about ranking sentences according to how useful/important they are as part of summary, we will consider here a par- ticular ranking scheme based on the probability of a sentence being part of summary under a given DOV, i.e., P(y|vvv), (1) where y denotes a given sentence, and vvv (v1, . …

Content truncated to 500 words