Improved Transcription and Indexing of Oral History Interviews for Digital Humanities Research

Improved Transcription and Indexing of Oral History Interviews for Digital Humanities Research Michael Gref1, Joachim K¨ohler1, Almut Leh2 1Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, 53757 Sankt Augustin, Germany 2Institute for History and Biography, University of Hagen, 58097 Hagen, Germany {michael.gref, joachim.koehler}@iais.fraunhofer.de, almut.leh@fernuni-hagen.de Abstract This paper describes different approaches to improve the transcription and indexing quality of the Fraunhofer IAIS Audio Mining system on Oral History interviews for the Digital Humanities Research. As an essential component of the Audio Mining system, automatic speech recognition faces a lot of difficult challenges when processing Oral History interviews. We aim to overcome these challenges using state-of-the-art automatic speech recognition technology. Different acoustic modeling techniques, like multi-condition training and sophisticated neural networks, are applied to train robust acoustic models. To evaluate the performance of these models on Oral History interviews a German Oral History test-set is presented. This test-set represents the large audio-visual archives “Deutsches Ged¨achtnis” of the Institute for History and Biography. The combination of the different applied techniques results in a word error rate reduced by 28.3% relative on this test-set compared to the current baseline system while only one eighth of the previous amount of training data is used. In context of these experiments new opportunities are set out for Oral History research offered by Audio Min- ing. Also the workflow is described used by Audio Mining to process long audio-files to automatically create time-aligned transcriptions. Keywords: acoustic modeling, robust speech recognition, multi-condition training, speech retrieval, oral history 1. Introduction The use of automatic speech recognition technology (ASR) to transcribe and index Oral History interviews has started with the MALACH project (Psutka et al., 2002) where the interviews of the Survivors of the Shoah Visual History Foundation (VHF) were processed with a state-of-the-art speech recognition in 2002. The main challenge of this activity was the variety and quality of the recordings and the variety of the languages. (Oard, 2012) investigated how speech recognition technology can be used for Oral History research. In comparison to other speech collections Oral History recordings have a lot challenges. Often the audio record- ing quality is low. Further the interviewed persons are el- derly persons speaking very spontaneous. This leads to poor recognition performance when applying off-the-shelf speech recognition technology. On the other hand we observe huge progress in speech recognition using different neural network learning frame- works. Recently many researchers, e.g. from Microsoft and IBM, have reported excellent results on conversational recognition tasks, like switchboard. Also with the open- source Kaldi ASR toolkit huge advances are reported us- ing different sophisticated neural network architecture and training methods. The Fraunhofer IAIS Audio Mining system is designed to automatically create segmented and time-aligned transcrip- tions from very long, unstructured audiovisual media files. In this work we want to present the opportunities and ad- vantages offered by the Audio Mining system for the Dig- ital Humanities Research by automatically creating tran- scriptions of Oral History interviews. Moreover we present the challenges for automatic speech recognition systems …

Content truncated to 500 words