Lessons Learned in Part-of-Speech Tagging of Conversational Speech

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 821–831, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Lessons Learned in Part-of-Speech Tagging of Conversational Speech Vladimir Eidelman†, Zhongqiang Huang†, and Mary Harper†‡ †Laboratory for Computational Linguistics and Information Processing Institute for Advanced Computer Studies University of Maryland, College Park, MD ‡Human Language Technology Center of Excellence Johns Hopkins University, Baltimore, MD {vlad,zhuang,mharper}@umiacs.umd.edu Abstract This paper examines tagging models for spon- taneous English speech transcripts. We ana- lyze the performance of state-of-the-art tag- ging models, either generative or discrimi- native, left-to-right or bidirectional, with or without latent annotations, together with the use of ToBI break indexes and several meth- ods for segmenting the speech transcripts (i.e., conversation side, speaker turn, or human- annotated sentence). Based on these studies, we observe that: (1) bidirectional models tend to achieve better accuracy levels than left-to- right models, (2) generative models seem to perform somewhat better than discriminative models on this task, and (3) prosody improves tagging performance of models on conversa- tion sides, but has much less impact on smaller segments. We conclude that, although the use of break indexes can indeed significantly im- prove performance over baseline models with- out them on conversation sides, tagging ac- curacy improves more by using smaller seg- ments, for which the impact of the break in- dexes is marginal. 1 Introduction Natural language processing technologies, such as parsing and tagging, often require reconfiguration when they are applied to challenging domains that differ significantly from newswire, e.g., blogs, twit- ter text (Foster, 2010), or speech. In contrast to text, conversational speech represents a significant challenge because the transcripts are not segmented into sentences. Furthermore, the transcripts are of- ten disfluent and lack punctuation and case informa- tion. On the other hand, speech provides additional information, beyond simply the sequence of words, which could be exploited to more accurately assign each word in the transcript a part-of-speech (POS) tag. One potentially beneficial type of information is prosody (Cutler et al., 1997). Prosody provides cues for lexical disambigua- tion, sentence segmentation and classification, phrase structure and attachment, discourse struc- ture, speaker affect, etc. Prosody has been found to play an important role in speech synthesis sys- tems (Batliner et al., 2001; Taylor and Black, 1998), as well as in speech recognition (Gallwitz et al., 2002; Hasegawa-Johnson et al., 2005; Ostendorf et al., 2003). Additionally, prosodic features such as pause length, duration of words and phones, pitch contours, energy contours, and their normalized val- ues have been used for speech processing tasks like sentence boundary detection (Liu et al., 2005). Linguistic encoding schemes like ToBI (Silver- man et al., 1992) have also been used for sentence boundary detection (Roark et al., 2006; Harper et al., 2005), as well as for parsing (Dreyer and Shafran, 2007; Gregory et al., 2004; Kahn et al., 2005). In the ToBI scheme, aspects of prosody such as tone, prominence, and degree of juncture between words are represented symbolically. …

Content truncated to 500 words