Mining the Spoken Wikipedia for Speech Data and Beyond

Mining the Spoken Wikipedia for Speech Data and Beyond Arne K¨ohn, Florian Stegen, Timo Baumann Department of Informatics Universit¨at Hamburg {koehn,1stegen,baumann}@informatik.uni-hamburg.de Abstract We present a corpus of time-aligned spoken data of Wikipedia articles as well as the pipeline that allows to generate such corpora for many languages. There are initiatives to create and sustain spoken Wikipedia versions in many languages and hence the data is freely available, grows over time, and can be used for automatic corpus creation. Our pipeline automatically downloads and aligns this data. The resulting German corpus currently totals 293h of audio, of which we align 71h in full sentences and another 86h of sentences with some missing words. The English corpus consists of 287h, for which we align 27h in full sentence and 157h with some missing words. Results are publically available.1 Keywords: Spoken Wikipedia, Long Audio Alignment, Speech Resource 1. Introduction Most time-aligned speech data for corpus analyses or train- ing models is non-free. This produces a barrier for research. In addition, models generated from these corpora cannot be freely distributed which also hinders research. The Spoken Wikipedia2, in contrast, is a large speech re- source under a free license with corresponding text available, covering a broad variety of topics. It is constantly evolving and of considerable size for several languages. While the (written) Wikipedia is already being widely used for research in computational linguistics (Atserias et al., 2008; Ahn et al., 2004; Nothman et al., 2009; Horn et al., 2014, and many others), the Spoken Wikipedia is not yet used in speech research, although it is a broad and multi- lingual source with lots of automatic and manual annotation available (such as links and topic relatedness) for the textual material. One major problem is the missing linkage between the spoken and written text as well as the semi-structured nature of Wikipedia data. We present a pipeline that aligns audio from the Spoken Wikipedia to the article text being read. We show that the data and process allow to bootstrap free speech recognition models from non-free ones (which we will publish with the full paper). The remainder of the paper is structured as follows: we present some descriptive statistics about the Spoken Wikipedia corpus in Section 2, highlighting the overall amounts of material that is available. We then describe some challenges about the available data in Section 3 and our pipeline for extracting and editing the downloadable data into a useful resource in Section 4. We describe the resulting data sets in Section 5, will describe our use of the data in Section 7 of the full paper and conclude with Section 8. 2. Spoken Wikipedia The Spoken Wikipedia is a project in which volunteers read out and record Wikipedia articles. One main aim is to make the information from the articles available through another 1nats-www.informatik.uni-hamburg.de/SWC/ 2http://en.wikipedia.org/wiki/Wikipedia: WikiProject_Spoken_Wikipedia; also contains links to other languages. modality, e. g. for visually impaired people, and for other hands- and eyes-free uses, such as while driving. The articles being …

Content truncated to 500 words