The ATIS Spoken Language Systems Pilot Corpus

The ATIS Spoken Language Systems Pilot Corpus Charles T. Hemphill, John J. Godfrey, George R. Doddington Texas Instruments Incorporated PO Box 655474, MS 238 Dallas, Texas 75265 Abstract Speech research has made tremendous progress in the past using the following paradigm: • define the research problem, • collect a corpus to objectively measure progress, and • solve the research problem. Natural language research, on the other hand, has typ- ically progressed without the benefit of any corpus of data with which to test research hypotheses. We de- scribe the Air Travel Information System (ATIS) pilot corpus, a corpus designed to measure progress in Spo- ken Language Systems that include both a speech and natural language component. This pilot marks the first full-scale attempt to collect such a corpus and provides guidelines for future efforts. Introduction The ATIS corpus provides an opportunity to develop and evaluate speech systems that understand sponta- neous speech. This corpus differs from its predecessor, the Resource Management corpus (Price eg al, 1988), in at least four significant ways. 1. Instead of being read, the speech has many of the characteristics of spontaneous spoken language (e.g., dysfiuencies, false starts, and colloquial pro- nunciations). 2. The speech collection occurs in an office environ- ment rather than a sound booth. 3. The grammar becomes part of the system under evaluation rather than a given part of the experi- ment. 4. The reference answer consists of the actual reply for the utterance rather than an orthographic tran- scription of the speech. The evaluation methodology supported by ATIS de- pends on having a comparable representation of the an- swer for each utterance. This is accomplished by limiting the utterances to database queries~ and the answers to a ground set of tuples from a fixed relational database. The ATIS corpus comprises the acoustic speech data for a query, transcriptions of that query, a set of tuples that constitute the answer, and the SQL expression for the query that produced the answer tuples. The ATIS database consists of data obtained from the Official Airline Guide (OAG, 1990), organized un- der a relational schema. The database remained fixed throughout the pilot phase. It contains information about flights, fares, airlines, cities, airports, and ground services, and includes twenty-five supporting tables. The large majority of the questions posed by subjects can be answered from the database with a single relational query. To collect the kind of English expected in a real work- ing system, we simulate one. The subject, or "travel planner," is in one room, with those running the simula- tion in another. The subject speaks requests over a mi- crophone and receives both a transcription of the speech and the answer on a computer screen. A session lasts approximately one hour, including detailed preliminary instructions and an exit questionnaire. Two "wizards" carry out the simulation: one tran- scribes the query while the other produces the answer. The transcriber interprets any verbal editing by the sub- ject and removes dysfluencies in order to produce an …

Content truncated to 500 words