AMISCO: The Austrian German Multi-Sensor Corpus

AMISCO: The Austrian German Multi-Sensor Corpus Hannes Pessentheiner, Thomas Pichler, Martin Hagm¨uller Signal Processing and Speech Communication Laboratory Graz University of Technology, Inffeldgasse 16c/EG, 8010 Graz, Austria hannes.pessentheiner@tugraz.at, thomas.pichler@student.tugraz.at, hagmueller@tugraz.at Abstract We introduce a unique, comprehensive Austrian German multi-sensor corpus with moving and non-moving speakers to facilitate the evaluation of estimators and detectors that jointly detect a speaker’s spatial and temporal parameters. The corpus is suitable for various machine learning and signal processing tasks, linguistic studies, and studies related to a speaker’s fundamental frequency (due to recorded glottograms). Available corpora are limited to (synthetically generated/spatialized) speech data or recordings of musical instruments that lack moving speakers, glottograms, and/or multi-channel distant speech recordings. That is why we recorded 24 spatially non-moving and moving speakers, balanced male and female, to set up a two-room and 43-channel Austrian German multi-sensor speech corpus. It contains 8.2 hours of read speech based on phonetically balanced sentences, commands, and digits. The orthographic transcriptions include around 53,000 word tokens and 2,070 word types. Special features of this corpus are the laryngograph recordings (representing glottograms required to detect a speaker’s instantaneous fundamental frequency and pitch), corresponding clean-speech recordings, and spatial information and video data provided by four Kinects and a camera. Keywords: Joint detection, joint estimation, direction of arrival, fundamental frequency, source localization, position-tracking, pitch-tracking, multi-channel, multi-room, multi-modal, glottogram, voice-controlled environment, Kinect, MEMS. 1. Introduction In computational auditory (Bregman, 1990; Wang and Brown, 2006) and acoustic scene (Teutsch, 2007) analysis, signal parameters often need to be associated with their ori- gin. For instance, to describe an acoustic scene, we need to detect and localize events, separate them from each other, characterize and interpret them (Imoto et al., 2013; Kwon et al., 2009; de Cheveign´e and Slama, 2006). To localize and characterize such events, we jointly detect or estimate each source’s parameters to avoid data association performed by additional algorithms. 1.1. Problem Definition In the area of distant speech enhancement (Vary and Mar- tin, 2006; Woelfel and McDonough, 2009), several research teams dedicated their time to jointly detect or estimate a source’s direction of arrival (DOA) and fundamental fre- quency (f0) with two or more microphones. Finding these parameters is a prerequisite to improve, e.g., the word ac- curacy rate of a speech recognizer by applying beamform- ing or source separation algorithms. They applied their algorithms to synthesized harmonic signals (Jensen et al., 2010; Kronvall et al., 2014), signals from musical instru- ments (Jensen et al., 2013; Jensen et al., 2015), certain parts of filtered clean-speech signals (Ngan et al., 2003; Karimian-Azari et al., 2013), synthetically spatialized sig- nals (Karimian-Azari et al., 2013), or speech signals with- out having a reliable ground truth of the f0 (Habib and Romsdorfer, 2013). One thing they all had in common was no access to multi-channel speech data recorded in real en- vironments and labeled with f0s and DOAs or positions in space, because such data did not exist. 1.2. Problem Solution To make such data publicly available, we, therefore, set up a …

Content truncated to 500 words