Balancing data-driven and rule-based approaches in the context of a Multimodal Conversational System

Balancing Data-driven and Rule-based Approaches in the Context of a Multimodal Conversational System Srinivas Bangalore AT&T Labs-Research 180 Park Avenue Florham Park, NJ 07932 srini@research.att.com Michael Johnston AT&T Labs-Research 180 Park Avenue Florham Park, NJ 07932 johnston@research.att.com Abstract Moderate-sized rule-based spoken language models for recognition and understanding are easy to develop and provide the ability to rapidly prototype conversational applications. However, scalability of such systems is a bot- tleneck due to the heavy cost of authoring and maintenance of rule sets and inevitable brittle- ness due to lack of coverage in the rule sets. In contrast, data-driven approaches are robust and the procedure for model building is usu- ally simple. However, the lack of data in a par- ticular application domain limits the ability to build data-driven models. In this paper, we ad- dress the issue of combining data-driven and grammar-based models for rapid prototyping of robust speech recognition and understanding models for a multimodal conversational sys- tem. We also present methods that reuse data from different domains and investigate the lim- its of such models in the context of a particular application domain. 1 Introduction In the past four decades of speech and natural language processing, both data-driven approaches and rule-based approaches have been prominent at different periods in time. In the recent past, rule-based approaches have fallen into disfavor due to their brittleness and the sig- nificant cost of authoring and maintaining complex rule sets. Data-driven approaches are robust and provide a simple process of developing applications given the data from the application domain. However, the reliance on domain-specific data is also one of the significant bottle- necks of data-driven approaches. Development of a con- versational system using data-driven approaches cannot proceed until data pertaining to the application domain is available. The collection and annotation of such data is extremely time-consuming and tedious, which is aggra- vated by the presence of multiple modalities in the user’s input, as in our case. Also, extending an existing applica- tion to support an additional feature requires adding ad- ditional data sets with that feature. In this paper, we explore various methods for combin- ing rule-based and in-domain data for rapid prototyping of speech recognition and understanding models that are robust to ill-formed or unexpected input in the context of a multimodal conversational system. We also investi- gate approaches to reuse out-of-domaindata and compare their performance against the performance of in-domain data-driven models. We investigate these issues in the context of a multi- modal application designed to provide an interactive city guide: MATCH. In Section 2, we present the MATCH application, the architecture of the system and the appa- ratus for multimodal understanding. In Section 3, we dis- cuss various approaches to rapid prototyping of the lan- guage model for the speech recognizer and in Section 4 we present two approaches to robust multimodal under- standing. Section 5 presents the results for speech recog- nition and multimodal understanding using the different approaches we consider. 2 The MATCH application MATCH (Multimodal Access To …

Content truncated to 500 words