Driving ROVER with Segment-based ASR Quality Estimation

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1095–1105, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Driving ROVER with Segment-based ASR Quality Estimation Shahab Jalalvand(1,2), Matteo Negri(1), Daniele Falavigna(1), Marco Turchi(1) (1) FBK - Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy (2) University of Trento, Italy {jalalvand,negri,falavi,turchi}@fbk.eu Abstract ROVER is a widely used method to combine the output of multiple auto- matic speech recognition (ASR) systems. Though effective, the basic approach and its variants suffer from potential draw- backs: i) their results depend on the order in which the hypotheses are used to feed the combination process, ii) when applied to combine long hypotheses, they disre- gard possible differences in transcription quality at local level, iii) they often rely on word confidence information. We address these issues by proposing a segment-based ROVER in which hypothesis ranking is obtained from a confidence-independent ASR quality estimation method. Our re- sults on English data from the IWSLT2012 and IWSLT2013 evaluation campaigns significantly outperform standard ROVER and approximate two strong oracles. 1 Introduction In automatic speech recognition (ASR), the com- bination of transcription hypotheses produced by multiple systems usually leads to significant word error rate (WER) reductions compared to the out- put of each individual system. Systems’ diversity and complementarity have been exploited in dif- ferent ways to synthetically obtain more accurate transcriptions. Recognizer output voting error re- duction – ROVER (Fiscus, 1997), the most widely used method, performs hypothesis fusion in two steps. First, the 1-best transcriptions from multi- ple systems are aligned by means of dynamic pro- gramming to build a single, minimal word tran- sition network. Then, the resulting network is searched to select the best scoring word at each node. The final hypothesis is constructed via a ma- jority voting mechanism and, if available, by using word confidence measures. This general strategy has been improved in sev- eral ways but, despite their proven effectiveness, ROVER and its variants have three potential draw- backs. The first one is intrinsic to their implemen- tation: the fusion process starts from one of the input hypotheses, which is used as “skeleton” for the greedy alignment of the others. The order in which the hypotheses are used to feed the process can hence determine significant variations in the WER of the resulting combination. This calls for automatic methods for ranking the hypotheses to initialise and carry on the fusion process. The second drawback is inherent to the way ROVER is usually run: the fusion process is typi- cally fed with transcriptions of entire audio record- ings (lasting up to hours). With this level of granu- larity, the skeleton used as basis for the alignment may consist of long transcriptions whose quality can considerably vary at local level. For instance, the worst transcription of an entire audio recording (globally) could be the best one for some passages (locally). This calls for solutions capable to op- erate …

Content truncated to 500 words