Using Document Summarization Techniques for Speech Data Subset Selection

Proceedings of NAACL-HLT 2013, pages 721–726, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Using Document Summarization Techniques for Speech Data Subset Selection Kai Wei∗, Yuzong Liu∗, Katrin Kirchhoff , Jeff Bilmes Department of Eletrical Engineering University of Washington Seattle, WA 98195, USA {kaiwei,yzliu,katrin,bilmes}@ee.washington.edu Abstract In this paper we leverage methods from sub- modular function optimization developed for document summarization and apply them to the problem of subselecting acoustic data. We evaluate our results on data subset selection for a phone recognition task. Our framework shows significant improvements over random selection and previously proposed methods us- ing a similar amount of resources. 1 Introduction Present-day applications in spoken language technol- ogy (speech recognizers, keyword spotters, etc.) can draw on an unprecedented amount of training data. However, larger data sets come with increased de- mands on computational resources; moreover, they tend to include redundant information as their size increases. Therefore, the performance gain curves of large-scale systems with respect to the amount of training data often show “diminishing returns”: new data is often less valuable (in terms of performance gain) when added to a larger pre-existing data set than when added to a smaller pre-existing set (e.g.,(Moore, 2003)). Therefore it is of prime importance to de- velop methods for data subset selection. We distin- guish two data subselection scenarios: (a) a priori selection of a data set before (re-)training a system; in this case the goal is to subselect the existing data set as well as possible, eliminating redundant infor- mation; (b) selection for adaptation, where the goal ∗These authors are joint first authors with equal contribu- tions. is to tune a system to a known development or test set. While many studies have addressed the second scenario, this paper investigates the first: our goal is to select a smaller subset of the data that fits a given ’budget’ (e.g. maximum number of hours of data) but provides, to the extent possible, as much information as the complete data set. Additionally, our selection method should be a low-resource method that does not require an already-trained complex system such as an existing word recognizer. This problem is akin to unsupervised data ’sum- marization’. In (Lin and Bilmes, 2009) a novel class of summarization techniques based on submodular function optimization were proposed for extractive document summarization. Interestingly, these meth- ods can also be applied to speech data ’summariza- tion’ with only small modifications. In the following sections we develop a submodular framework for speech data summarization and evaluate it on a proof- of-concept phone recognition task. 2 Related Work Most approaches to data subset selection in speech have relied on “rank-and-select” approaches that de- termine the utility of each sample in the data set, rank all samples according to their utility scores, and then select the top N samples. In weakly supervised approaches (e.g.,(Kemp and Waibel, 1998; Lamel et al., 2002; Hakkani-Tur et al., 2002), utility is re- lated to the confidence of an existing word recognizer on new data samples: …

Content truncated to 500 words