Hidden Understanding Models of Natural Language

HIDDEN UNDERSTANDING MODELS OF NATURAL LANGUAGE Scott Miller College of Computer Science Northeastern University Boston, MA 02115 millers@ccs.neu.edu Robert Bobrow, Robert Ingria, Richard Schwartz BBN Systems and Technologies 70 Fawcett St., Cambridge, MA 02138 rusty, ingria, schwartz@BBN.COM Abstract We describe and evaluate hidden understanding models, a statistical learning approach to natural language understanding. Given a string of words, hidden understanding models determine the most likely meaning for the string. We discuss 1) the problem of representing meaning in this framework, 2) the structure of the statistical model, 3) the process of training the model, and 4) the process of understanding using the model. Finally, we give experimental results, including results on an ARPA evaluation. 1 Introduction Hidden understanding models are an innovative class of statistical mechanisms that, given a string of words, determines the most likely meaning for the string. The overall approach represents a substantial departure from traditional techniques by replacing hand-crafted grammars and rules with statistical models that are automatically learned from examples. Hidden understanding models were primarily motivated by techniques that have been extremely successful in speech recognition, especially hidden Markov models [Baum, 72]. Related techniques have previously been applied to the problem of identifying concept sequences within a sentence [Pieraccini et aL, 91]. In addition, the approach contains elements of other natural language processing techniques including semantic grammars [Waltz, 78; Hen&ix, 78], augmented transition networks (ATNs) [Woods, 70], probabilistic parsing [Fujisaki et al., 89; Chitrao and Grishman, 90; Seneff, 92], and automatic grammar induction [Pereira and Schabes, 92]. Hidden understanding models are capable of learning a variety of meaning representations, ranging from simple domain-specific representations, to ones at a level of detail and sophistication comparable to current natural language systems. In fact, a hidden understanding model can be used to produce a representation with essentially the same information content as the semantic graph used by the Delphi system [-Bobrow et al., 90], a general purpose NLP system, which utilizes a modified Definite Clause Grammar formalism. This fact made it possible to interface a hidden understanding system to the discourse processing and data- base retrieval components of Delphi to produce a complete "end to end" system. This hybrid system participated in the 1993 ATIS natural language evaluation. Although only four months old, the scores achieved by the combined system were quite respectable. Because of differences between language understanding and speech recognition, significant changes are required in the hidden Markov model methodology. Unlike speech, where each phoneme results in a local sequence of spectra, the relation between the meaning of a sentence and the sequence of words is not a simple linear sequential model. Language is inherently nested, with subgroups of concepts within other concepts. A statistical system for understanding language must take this and other differences into account in its overall design. In principle, we have the following requirements for a hidden understanding system: • A notational system for expressing meanings. • A statistical model that is capable of representing meanings and the association between meanings and …

Content truncated to 500 words