Collection and Analyses of WSJ-CSR Data at MIT

Collection and Analyses of WSJ-CSR Data at MIT 1 Michael Phillips, James Glass, Joseph Polifroni, and Victor Zue Spoken Language Systems Group Laboratory for Computer Science Massachusetts Institute of Technology Cambridge, Massachusetts 02139 ABSTRACT Recently, the DARPA community started a new data col- lection initiative in the Wall Street Journal (WSJ) domain to support research and development of very large vocabu- lary continuous speech recognition (CSR) systems. Since Au- gust 1991, our group has actively participated in the develop- ment of the WSJ-CSR corpus. The purpose of this paper is to document our involvement in this process, from recording and transcription to analyses and distribution. We will also present the results of an experiment investigating the prepro- cessing of the prompt text. INTRODUCTION One of the key ingredients that has contributed to the steady improvement in speech recognition technol- ogy in recent years is the availability of large speech cor- pora [1,3,7,8]. With the help of these corpora, researchers have been able to develop recognition systems and obtain reliable estimates of system parameters. Perhaps just as important, these corpora, together with standardized performance evaluation procedures and metrics, have en- couraged objective comparison of different systems, lead- ing to better understanding and cross fertilization of re- search ideas [4]. The various speech corpora that the DARPA commu- nity has collected serve a wide range of purposes. The TIMIT corpus was designed with acoustic-phonetic re- search in mind. The Resource Management corpus ad- dresses the needs for developing recognition systems with moderate vocabulary (1,000 words) and perplexity (60, with a word-pair language model). The VOYACER and ATIS corpora contain spontaneously generated speech, and are useful for spoken language system development. All the presently available corpora have moderate vocab- ulary sizes and perplexities, and thus cannot adequately support research and development of very large vocab- ulary continuous speech recognition (CSR) systems in American English 2. As a result, the DARPA community 1This research was supported by DARPA under Contract N00014-89-J-1332, monitored through the Office of Naval Research. 2A large corpus of spoken French has recently been collected by 367 recently initiated an effort towards the construction of a new corpus to meet these needs. The domain chosen by the community is the Wall Street Journal (WSJ), and the text prompts are selected from the CD-ROM distributed by ACL/DCI [5]. While the ultimate goal is to collect around 300 hours of speech from more than 100 speakers, it was thought that we should collect a pilot corpus of approximately 40 hours, partly to satisfy near term needs and partly to debug the text preprocessing and data collection processes. Since August 1991, our group is one of three that actively par- ticipated in the collection of the WSJ-CSR pilot corpus 3. The purpose of this paper is to document our involvement in this process, present some comparative analyses of the resulting data, and describe an experiment investigating the preparation of the prompt text. DATA COLLECTION The Environment All the MIT data are collected in an …

Content truncated to 500 words