NAtural Language driven Image Generation

NAtural Language driven Image Generation Giovanni Adorni, Mauro Di Manzo and Fausto Giunchiglis Department of Communication, Computer and System Sciences University of Genoa Via Opera Pia i] A - 16145 Genoa - Italy ABSTRACT In this paper the experience made through the development of a NAtural Language driven Image Generation is discussed. This system is able to imagine a static scene described by means of a sequence of simple phrases. In particular, a theory for equilibrium and support will be outlined together with the problem of object positioning. i. IntrOduction A challenging application of the AI techniques is the generation of 2D projections of 3D scenes starting from a possibly unformalized input, as a natural language description. Apart from the practically unlimited simulation capabilities that a tool of this kind could give people working in the show business, a better modeling of the involved cognitive processes is important not only from the point of view of story understanding (Wa8Oa,WaS]a), but also for a more effective approach to a number of AI related problems, as, for instance, vision or robot planning (So76a). In this paper we discuss some of the ideas on which is based a NAtural Language driven Image Generation (NALIG from here on) which has been developed for experimental purposes at the University of Genoa. This system is currently able to reason about static scenes described by means of a set of simple phrases of the form: csubject~ ~preposition~ cobject, [ creference~ ] (*). The understanding process in NALIG flows through several steps (distinguishable only from a logic point of view), which perform object instantiation, relation inheritance, translation of the surface expression into unambiguous primitives, (*) NALIG has been developed for the Italian language; the prepositions it can presently analyze are: su, sopra, sotto, a destra, a sinistra, vici- no, davanti, dietro, in. A second deeply revised release is currently under design. This work has been supported by the Italian Depart- ment of Education under Grant M.P.I.-27430. consistency checking, object positioning and so on, up to the drawing of the "imagined" scene on a screen. A general overview of NALIG is given in the paper, which however is mainly concerned with the role of common sense physical reasoning in consistency checking and object instantiation. Qualitative reasoning about physical processes is a promising tool which is exciting the interest of an increasing number of A.I. researches (Fo83a,Fo83b,Fo83c) , (Ha78a,Ha79a) , (K179a,K183a). It plays a central role in the scene description understanding process for several reasons: i. naive physics, following Hayes definition (Ha78a), is an attempt to represent the common sense knowledge that people have about the physical world. Sharing this knowledge between the speaker and the listener (the A.I. system, in our case) is the only feasible way to let the second make realistic hypotheses about the assumptions underlying the speaker utterances; ii. it allows to reach conclusions about problems for which very little information is available and which consequently are hard to formalize using quantitative models; iii. qualitative reasoning …

Content truncated to 500 words