Repurposing Corpora for Speech Repair Detection: Two Experiments

Repurposing Corpora for Speech Repair Detection: Two Experiments Simon Zwarts, Mark Johnson, Robert Dale Center for Language Technology, Department of Computing, Macquarie University {simon.zwarts, mark.johnson, robert.dale}@mq.edu.au Abstract Unrehearsed spoken language often contains many disfluencies. If we want to correctly interpret the content of spoken language, we need to be able to detect these disfluencies and deal with them appropriately. In the work de- scribed here, we use a statistical noisy channel model to detect disfluencies in transcripts of spoken language. Like all statistical approaches, this is natu- rally very data-hungry; however, cor- pora containing transcripts of unre- hearsed spoken language with disflu- encies annotated are a scarce resource, which makes training difficult. We address this issue in the follow- ing ways: First, since written textual corpora are much more abundant than speech corpora, we see whether using a large text corpus to increase the data available to our language model com- ponent delivers an improvement. Sec- ond, given that most spoken language corpora are not annotated with disflu- encies, we explore the use of Expecta- tion Maximisation to mark the disflu- encies in such corpora, so as to increase the data availability for our complete model. In neither case do we see an improve- ment in our results. We discuss these results and the possible reasons for the negative outcome. 1 Introduction We are interested in improving speech disflu- ency detection in transcripts of spontaneous spoken language. Many models have been pro- posed for this task in the literature; the best performing models so far are statistical by na- ture and have large data needs. A statistical natural language processing al- gorithm typically has two important compo- nents: a model that describes the behaviour of interest, and the training data which is neces- sary to guide that model. It has been observed that simple algorithms can outperform more complex models when these simple algorithms have the advantage in terms of the amount of data available; so, for example, Brill and Banko (2001) argue that more data is more im- portant than better algorithms for some natu- ral language processing tasks. It is this insight that drives the work described in this paper. Our current approach to speech disfluency detection is trained on manually-constructed spoken language corpora which contain anno- tations of all disfluencies as part of the tran- scription process. Our model is based on the noisy channel model and consists of a language model and a channel model. As we have re- ported elsewhere (Zwarts et al., 2010), we are able to achieve reasonable results when using Switchboard data: we obtain an F-score of 0.757 in determining which constituents of an utterance belong to a disfluency. We would like to see if we can improve on our previously reported performance by adding more data. Our language model does not need any special annotation, and so our first set of experiments investigates whether we can improve results by vastly increasing the training data for the language model. The task of …

Content truncated to 500 words