Learning Perceptually-Grounded Semantics in the L₀ Project

LEARNING PERCEPTUALLY-GROUNDED SEMANTICS IN THE L0 PROJECT Terry Regier* International Computer Science Institute 1947 Center Street, Berkeley, CA, 94704 (415) 642-4274 x 184 regier@cogsci.Berkeley.ED U • TR "Above" Figure 1: Learning to Associate Scenes with Spatial Terms ABSTRACT A method is presented for acquiring perceptually- grounded semantics for spatial terms in a simple visual domain, as a part of the L0 miniature language acquisi- tion project. Two central problems in this learning task are (a) ensuring that the terms learned generalize well, so that they can be accurately applied to new scenes, and (b) learning in the absence of explicit negative ev- idence. Solutions to these two problems are presented, and the results discussed. 1 Introduction The L0 language learning project at the International Computer Science Institute [Feldman et al., 1990; We- ber and Stolcke, 1990] seeks to provide an account of lan- guage acquisition in the semantic domain of spatial rela- tions between geometrical objects. Within this domain, the work reported here addresses the subtask of learn- ing to associate scenes, containing several simple objects, with terms to describe the spatial relations among the objects in the scenes. This is illustrated in Figure 1. For each scene, the learning system is supplied with an indication of which object is the reference object (we call this object the landmark, or LM), and which object is the one being located relative to the reference object (this is the trajector, or TR). The system is also supplied with a single spatial term that describes the spatial relation *Supported through the International Computer Science Institute. portrayed in the scene. It is to learn to associate all applicable terms to novel scenes. The TR is restricted to be a single point for the time being; current work is directed at addressing the more general case of an arbitrarily shaped TR. Another aspect of the task is that learning must take place in the absence of explicit negative instances. This condition is imposed so that the conditions under which learning takes place will be similar in this respect to those under which children learn. Given this, there are two central problems in the sub- task as stated: • Ensuring that the learning will generalize to scenes which were not a part of the training set. This means that the region in which a TR will be consid- ered "above" a LM may have to change size, shape, and position when a novel LM is presented. • Learning without explicit negative evidence. This paper presents solutions to both of these prob- lems. It begins with a general discussion of each of the two problems and their solutions. Results of training are then presented. Then, implementation details are discussed. And finally, some conclusions are presented. 2 Generalization and Parameterized Regions 2.1 The Problem The problem of learning whether a particular point lies in a given region of space is a foundational one, with sev- eral widely-known "classic" solutions [Minsky and Pa- pert, 1988; Rumelhart and McClelland, 1986]. …

Content truncated to 500 words