Multi-language Speech Collection for NIST LRE

Multi-language Speech Collection for NIST LRE Karen Jones, Stephanie Strassel, Kevin Walker, David Graff, Jonathan Wright Linguistic Data Consortium, University of Pennsylvania Philadelphia, PA 19104 USA Email:{karj,strassel,walkerk,graff,jdwright}@ldc.upenn.edu Abstract The Multi-language Speech (MLS) Corpus supports NIST’s Language Recognition Evaluation series by providing new conversational telephone speech and broadcast narrowband data in 20 languages/dialects. The corpus was built with the intention of testing system performance in the matter of distinguishing closely related or confusable linguistic varieties, and careful manual auditing of collected data was an important aspect of this work. This paper lists the specific data requirements for the collection and provides both a commentary on the rationale for those requirements as well as an outline of the various steps taken to ensure all goals were met as specified. LDC conducted a large-scale recruitment effort involving the implementation of candidate assessment and interview techniques suitable for hiring a large contingent of telecommuting workers, and this recruitment effort is discussed in detail. We also describe the telephone and broadcast collection infrastructure and protocols, and provide details of the steps taken to pre-process collected data prior to auditing. Finally, annotation training, procedures and outcomes are presented in detail. Keywords: language recognition; speech; telephone; broadcast 1. Introduction The Multi-Language Speech (MLS) Corpus was created by Linguistic Data Consortium (LDC) to support the ongoing Language Recognition Evaluation (LRE) campaign conducted by the National Institute of Standards and Technology (NIST). With speech data originating from over 9000 speakers, the MLS Corpus is roughly double the size of the last corpus developed by LDC for this evaluation series, the LRE11 Corpus (Strassel 2012). The current MLS Corpus is large enough to provide data for multiple LRE evaluation cycles. The MLS Corpus was most recently used in support of the LRE15 evaluation (NIST 2015), providing speech data in twenty linguistic varieties including 6600 segments manually verified for language. As with the LRE11 Corpus, the MLS Corpus was designed to support the evaluation of system capabilities for distinguishing closely related or confusable linguistic varieties. To support this goal we collected and audited audio for twenty distinct linguistic varieties across six defined “language clusters”, where the varieties within a given cluster can be considered mutually intelligible and/or typologically related to some degree. The MLS Corpus required collection of new conversational telephone speech (CTS) and/or broadcast narrowband speech (BNBS) for each language, as well as updates to the procedures for selecting and preparing test segments for inclusion in the LRE15 evaluation. In the sections that follow we describe construction of the MLS Corpus in detail. 2. Data Requirements The goal behind construction of the MLS Corpus was to collect narrowband speech from 400 unique speakers in each of 20 languages. Collection included two genres: conversational telephone speech (CTS) conversations between people who know one another, and broadcast narrowband speech (BNBS) taken from listener call-ins, person-on-the-street interviews or other instances where telephone data is embedded in a broadcast recording The twenty languages selected for inclusion in the MLS corpus were chosen from an …

Content truncated to 500 words