SCANMail: Audio Navigation in the Voicemail Domain

SCANMail: Audio Navigation in the Voicemail Domain Michiel Bacchiani Julia Hirschberg Aaron Rosenberg Steve Whittaker Donald Hindle Phil Isenhour Mark Jones Litza Stark Gary Zamchick fmichiel,julia,aer,stevewg@research.att.com, dhindle@answerlogic.com, isenhour@vt.edu, jones@research.att.com, litza@udel.edu, zamchick@attlabs.att.com AT&T Labs – Research 180 Park Avenue Florham Park, NJ 07932-0971, USA ABSTRACT This paper describes SCANMail, a system that allows users to browse and search their voicemail messages by content through a GUI. Content based navigation is realized by use of automatic speech recognition, information retrieval, information extraction and human computer interaction technology. In addition to the browsing and querying functionalities, acoustics-based caller ID technology is used to proposes caller names from existing caller acoustic models trained from user feedback. The GUI browser also provides a note-taking capability. Comparing SCANMail to a regu- lar voicemail interface in a user study, SCANMail performed better both in terms of objective (time to and quality of solutions) as well as subjective objectives. 1. INTRODUCTION Increasing amounts of public, corporate, and private audio present a major challenge to speech, information retrieval, and human- computer interaction research: how can we help people to take ad- vantage of these resources when current techniques for navigating them fall far short of text-based search methods? In this paper, we describe SCANMail, a system that employs automatic speech recognition (ASR), information retrieval (IR), information extrac- tion (IE), and human computer interaction (HCI) technology to per- mit users to browse and search their voicemail messages by content through a GUI interface. A CallerId server also proposes caller names from existing caller acoustic models and is trained from user feedback. An Email server sends the original message plus its ASR transcription to a mailing address specified in the user’s profile. The SCANMail GUI also provides note-taking capabilities as well as browsing and querying features. Access to messages and information about them is presented to the user via a Java applet running under Netscape. Figure 1 shows the SCANMail GUI. . 2. SYSTEM DESCRIPTION In SCANMail, messages are first retrieved from a voicemail server, then processed by the ASR server that provides a transcription. The message audio and/or transcription are then passed to the IE, IR, Email, and CallerId servers. The acoustic and language model of the recognizer, and the IE and IR servers are trained on 60 hours of a 100 hour voicemail corpus, transcribed and hand labeled for tele- phone numbers, caller names, times, dates, greetings and closings. The corpus includes approximately 10,000 messages from approx- imately 2500 speakers. About 90% of the messages were recorded from regular handsets, the rest from cellular and speaker-phones. The corpus is approximately gender balanced and approximately 12% of the messages were from non-native speakers. The mean duration of the messages was 36.4 seconds; the median was 30.0 seconds. 2.1 Automatic Speech Recognition The baseline ASR system is a decision-tree based state-clustered triphone system with 8k tied states. The emission probabilities of the states are modeled by 12 component Gaussian mixture distribu- tions. The system uses a 14k vocabulary, automatically generated by …

Content truncated to 500 words