Integrating Symbolic and Statistical Approches in Speech and Natural Language Applications

Integrating Symbolic and Statistical Approaches in Speech and Natural Language Applications Marie Meteer & Herbert Gish BBN Systems & Technologies Cambridge, Massachusetts mmeteer@bbn.com hgish@bbn.com ABSTRACT Symbolic and statistical approaches have traditionally been kept separate and applied to very different problems. Symbolic techniques apply best where we have a priori knowledge of the language or the domain and where the application of a theory or study of selected examples can help leverage and extend our knowledge. Statistical approaches apply best where the results of decisions can be represented in a model and where we have sufficient training data to accurately estimate the parameters of the model. Another factor in selecting which approach to use in a particular situation is whether there is sufficient uncertainty to warrant the need to make educated guesses (statistical approach) rather than assertions (symbolic approach). In our work in gisting, word spotting, and topic classification, we have successfully integrated symbolic and statistical approaches in a range of tasks, including language modeling for speech recognition, information extraction from speech, and topic and event spotting. In this paper we outline the contributions and drawbacks of each approach and illustrate our points with the various components of our systems. 1. INTRODUCTION Symbolic and statistical approaches have both made significant contributions in speech and natural language processing. However, they have traditionally been kept separate and applied to very different kinds of problems. Most speech recognition systems use statistical techniques exclusively, whereas natural language (NL) systems are mostly symbolic. We are seeing more integration of statistical methods in NL, but usually in some well defined component, such as a statistically based part of speech tagger as a preprocessor to parsing. In this paper, we have two goals: first, to characterize the kinds of problems that are most amenable to each of these approaches and, second, to show how we have integrated the approaches in our work in information extraction from speech, topic classification, and word and phrase spotting. We begin with a brief overview characterizing the two approaches, then discuss in more detail how we have integrated these two approaches in our work. 1.1 Characterizing symbolic and statistical approaches Symbolic approaches have dominated work in NL. By writing rules, we can take advantage of what we already know about a language or domain and we can apply a theoretical framework or study of selected examples to leverage and extend our knowledge. Most symbolic approaches also have meaningful intermediate structures that indicate what steps a system goes through in processing. Furthermore, since in a rule based approach the system either works or fails (as opposed to being more or less likely as is the case in a statistical approach), we generally have a clearer understanding of what a system is capable of and where its weaknesses lie. However, this feature is also the greatest flaw of this kind of approach, as it makes a system brittle. Statistical approaches begin with a model and estimate the parameters of the model based on data. Since …

Content truncated to 500 words