A Skip-Chain Conditional Random Field for Ranking Meeting Utterances by Importance

Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 364–372, Sydney, July 2006. c⃝2006 Association for Computational Linguistics A Skip-Chain Conditional Random Field for Ranking Meeting Utterances by Importance∗ Michel Galley Columbia University Department of Computer Science New York, NY 10027, USA galley@cs.columbia.edu Abstract We describe a probabilistic approach to content se- lection for meeting summarization. We use skip- chain Conditional Random Fields (CRF) to model non-local pragmatic dependencies between paired utterances such as QUESTION-ANSWER that typi- cally appear together in summaries, and show that these models outperform linear-chain CRFs and Bayesian models in the task. We also discuss dif- ferent approaches for ranking all utterances in a se- quence using CRFs. Our best performing system achieves 91.3% of human performance when evalu- ated with the Pyramid evaluation metric, which rep- resents a 3.9% absolute increase compared to our most competitive non-sequential classifier. 1 Introduction Summarization of meetings faces many challenges not found in texts, i.e., high word error rates, ab- sence of punctuation, and sometimes lack of gram- maticality and coherent ordering. On the other hand, meetings present a rich source of structural and pragmatic information that makes summariza- tion of multi-party speech quite unique. In par- ticular, our analyses of patterns in the verbal ex- change between participants found that adjacency pairs (AP), a concept drawn from the conver- sational analysis literature (Schegloff and Sacks, 1973), have particular relevance to summarization. APs are pairs of utterances such as QUESTION- ANSWER or OFFER-ACCEPT, in which the second utterance is said to be conditionally relevant on the first. We show that there is a strong correlation be- tween the two elements of an AP in summariza- tion, and that one is unlikely to be included if the other element is not present in the summary. Most current statistical sequence models in nat- ural language processing (NLP), such as hidden ∗This material is based on research supported in part by the U.S. National Science Foundation (NSF) under Grants No. IIS-0121396 and IIS-05-34871, and the Defense Ad- vanced Research Projects Agency (DARPA) under Contract No. HR0011-06-C-0023. Any opinions, findings and con- clusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the NSF or DARPA. Markov models (HMMs) (Rabiner, 1989), are lin- ear chains that only encode local dependencies between utterances to be labeled. In multi-party speech, the two elements of an AP are gener- ally arbitrarily distant, and such models can only poorly account for dependencies underlying APs in summarization. We use instead skip-chain se- quence models (Sutton and McCallum, 2004), which allow us to explicitly model dependencies between distant utterances, and turn out to be par- ticularly effective in the summarization task. In this paper, we compare two types of network structures—linear-chain and skip-chain—and two types of network semantics—Bayesian Networks (BNs) and Conditional Random Fields (CRFs). We discuss the problem of estimating the class posterior probability of each utterance in a se- quence in …

Content truncated to 500 words