Acquiring Core Meanings of Words, Represented as Jackendoff-Style Conceptual Structures, From Correlated Streams of Linguistic and Non-Linguistic Input

ACQUIRING CORE MEANINGS OF WORDS, REPRESENTED AS JACKENDOFF-STYLE CONCEPTUAL STRUCTURES, FROM CORRELATED STREAMS OF LINGUISTIC AND NON-LINGUISTIC INPUT Jeffrey Mark Siskind* M. I. T. Artificial Intelligence Laboratory 545 Technology Square, Room NE43-800b Cambridge MA 02139 617/253-5659 internet: Qobi~AI.MIT.EDU Abstract This paper describes an operational system which can acquire the core meanings of words without any prior knowledge of either the category or meaning of any words it encounters. The system is given as input, a description of sequences of scenes along with sentences which describe the [EVENTS] taking place as those scenes unfold, and produces as out- put, a lexicon consisting of the category and mean- ing of each word in the input, that allows the sen- tences to describe the [EVENTS]. It is argued, that each of the three main components of the system, the parser, the linker and the inference component, make only linguistically and cognitively plausible assump- tions about the innate knowledge needed to support tractable learning. The paper discusses the theory underlying the system, the representations and al- gorithms used in the implementation, the semantic constraints which support the heuristics necessary to achieve tractable learning, the limitations of the current theory and the implications of this work for language acquisition research. 1 Introduction Several natural language systems have been reported which learn the meanings of new words[5, 7, 1, 16, 17, 13, 14]. Many of these systems (in particular [5, 7, 1]) learn the new meanings based upon expec- tations arising from the morphological, syntactic, se- *Supported by an AT&T Bell Laboratories Ph.D. scholar- ship. Part of this research was performed while the author was visiting Xerox PARC as a research intern and as a consultant. mantic and pragmatic context of the unknown word in the text being processed. For example, if such a system encounters the sentence "I woke up yesterday, turned off my alarm clock, took a shower, and cooked myself two grimps for breakfast[5]" it might conclude that grimps is a noun which represents a type of food. Such systems succeed in learning new words only when the context offers sufficient constraint to narrow down the possible meanings to make the ac- quisition unambiguous. Accordingly, such a theory accounts only for the type of learning which arises when an adult encounters an unknown word while reading a text comprised mostly of known words. It can not explain the kind of learning which a young child performs during the early stages of language acquisition when it starts out knowing the meanings of few if any words. In this paper, I present a new theory which can account for the language learning which a child ex- hibits. In this theory, the learner is presented with a training session consisting of a sequence of sce- narios. Each scenario contains both linguistic and non-linguistic (i.e. visual) information. The non- linguistic information for each scenario consists of a time-ordered sequence of scenes, each depicted via a conjunction of true and negated atomic formulas describing that scene. Likewise, the …

Content truncated to 500 words