Collection of Spontaneous Speech for the ATIS Domain and Comparative Analyses of Data Collected at MIT and TI

Collection of Spontaneous Speech for the ATIS Domain and Comparative Analyses of Data Collected at MIT and TI 1 Joseph Polifroni, Stephanie Seneff, and Victor W. Zue Spoken Language Systems Group Laboratory for Computer Science Massachusetts Institute of Technology Cambridge, Massachusetts 02139 ABSTRACT As part of our development of a spoken language system in the ATIS domain, we have begun a smail-scale effort in collecting spontaneous speech data. Our procedure differs from the one used at 'i~xas Instruments (TI) in many respects, the most important being the reliance on an existing system, rather than a wizard, to participate in data collection. Over the past few months, we have collected over 3,600 spontaneously generated sentences from 100 subjects. This paper documents our data collection process, and makes some comparative anaiyses of our data with those collected at TI. The advantages as well as disadvantages of this method of data collection will be discussed. INTRODUCTION AWLS, or Air Travel Information Service, is the desig- nated common task of the DARPA Spoken Language Sys- tems (SLS) Program [8]. As part of our development of a spoken language system in this domain, we have recently be- gun a small-scale effort in collecting spontaneous speech data. This effort is motivated partly by our desire to contribute to the data collection efforts already underway elsewhere [4,2,1], so that more data can be available to the community sooner for system development, training, and evaluation. In addi- tion, we were interested in exploring various alternatives of the data collection procedure itself. It is our belief that we as a community do not fully understand how goal-directed spon- taneous speech should best be collected. This is not surpris- ing, since we have little experience in this area. Nevertheless, data collection is an impori~ant area of research for the SLS Program, since the type of data that we collect will directly affect the capabilities of systems that we develop, and the evaluations that we can perform. Therefore, we thought it would be appropriate to experiment with different aspects of this process. There is evidence that even very small changes in the procedure, such as the instructions to the subject, can drastically alter the nature of the data collected [l]. The paper is organized as follows. We will first discuss 1This research was supported by DARPA under Contract N00014- 89-.11-1332, monitored through the Office of Naval Research. some methodological considerations that led to the particular collection procedure that we adopted. We will then briefly describe the procedure itself. This will be followed by some comparative analyses of a subset of the data that we have collected with those collected at Texas Instruments (TI). Im- plications of our findings will be discussed. DATA COLLECTION As is the case with other efforts [4~2,1], our data are col- lected under simulation. Nevertheless, we wanted the simula- tion to reflect as much as possible the system that we are de- veloping. In this section, we will briefly describe some deslgn issues and document the actual …

Content truncated to 500 words