VAST: A Corpus of Video Annotation for Speech Technologies

VAST: A Corpus of Video Annotation for Speech Technologies Jennifer Tracey, Stephanie Strassel Linguistic Data Consortium, University of Pennsylvania 3600 Market Street, Suite 810, Philadelphia, PA. 19104 USA {garjen, strassel}@ldc.upenn.edu Abstract The Video Annotation for Speech Technologies (VAST) corpus contains approximately 2900 hours of video data collected and labeled to support the development of speech technologies such as speech activity detection, language identification, speaker identification, and speech recognition. The bulk of the data comes from amateur video content harvested from the web. Collection was designed to ensure that the videos cover a diverse range of communication domains, data sources and video resolutions and to include three primary languages (English, Mandarin Chinese and Arabic) plus supplemental data in 7 additional languages/dialects to support language recognition research. Portions of the collected data were annotated for speech activity, speaker identity, speaker sex, language identification, diarization, and transcription. A description of the data collection and each of the annotation types is presented in this paper. The corpus represents a challenging data set for language technology development due to the informal nature of the majority of the data, as well as the variety of languages, noise conditions, topics, and speakers present in the collection. Keywords: speech corpora, video corpora, multilingual resources 1. Introduction The Video Annotation for Speech Technologies (VAST) corpus is the result of an effort to collect video content covering a diverse range of communication domains, data sources and video resolutions for use in training, development and testing of multiple speech technologies. The corpus comprises approximately 2900 hours of data, targeting three primary languages (English, Mandarin Chinese and Arabic) plus supplemental data in 7 additional languages/dialects to support language recognition research. Portions of the collected data were annotated for speech activity, speaker identity, speaker sex and language, diarization, and transcription. The collection of English, Mandarin, and Arabic data is referred to here as the “main corpus,” and the data from supplemental languages as well as the annotations are referred to as the “sub-corpora.” 2. Main corpus Videos in the main corpus contain speech in English (including US, UK, and other varieties), Mandarin Chinese, and Arabic. Videos are assigned a primary language designation, but some videos may contain more than one language due to codeswitching in naturally occurring data. The majority of data in the VAST corpus consists of amateur videos harvested from the Internet. In addition, a small amount of audio from broadcast television news and/or informal talk shows is included in the main corpus. No annotations were performed on the broadcast data, and the discussion of the data in this paper therefore focuses primarily on the amateur video. Criteria for inclusion of videos in the main corpus are as follows: • Videos must contain speech in one of the three primary languages • Any variety/dialect of Arabic or English is acceptable for the main corpus, while Chinese videos must contain Mandarin • Multi-party, informal speech is preferred over monologs, telephone-style dialogs or interviews • There is no restriction on topic (variety of …

Content truncated to 500 words