One event, many representations. Mapping action concepts through visual features.

One event, many representations. Mapping action concepts through visual features. Alessandro Panunzi, Andrea Amelio Ravelli, Lorenzo Gregori Universit`a degli Studi di Firenze, Florence, Italy {alessandro.panunzi, lorenzo.gregori, andreaamelio.ravelli}@unifi.it Abstract This paper faces the problem of unifying the representation of actions and events in different semantic resources. The proposed solution exploits the IMAGACT visual component (video scenes that represent physical actions) as the linkage point among resources. By using visual objects, we connected resources responding to different scopes and theoretical frameworks, in which a concept-to-concept mapping appeared difficult to obtain. We provide a brief description of two experiments that exploit IMAGACT videos as a linkage point: an automatic linking with BabelNet, a multilingual semantic network, and a manual linking with Praxicon, a conceptual knowledge base of action. The aim of this work is to integrate data from resources with different level of granularity in order to describe the action semantics from a linguistic, visual and motor point of view. Keywords: ontology linking, semantics, action verbs, knowledge representation 1. Introduction Action verb interpretation is a basic issue for human- machine interaction systems that aim to process natural language instructions. The difficulty behind automatic ac- tion verb understanding comes out from the evidence that no one-to-one correspondence can be established between action predicates (lexical items in each natural language) and action concepts (mental representations of experienced events). The same action can be predicated by multiple verbs and, conversely, one verb can extend to multiple and different actions. Most of these verbs belong to the class of general verbs, which are characterized by a high ambi- guity and high frequency in the use (Moneglia, 2010). In these circumstances, senses are often vague and overlap- ping, their discrimination is not clear, and this is a critical issue for their semantic representation. In fact, if we look at WordNet (Fellbaum, 1998), the proposed classification of general action verbs highlights two main issues: on one hand, a synset often encodes a variety of events that are cog- nitively conceived as different action concepts; on the other hand, it’s frequent that a specific event is not clearly de- scribed by a unique verb sense, but it seems to be spread on more senses (belonging to different synsets), each one rep- resenting a possible conceptualization of the same event. The following examples explain these classification prob- lems. Example 1 shows a synset that represents a very general sense of putting objects in a location that can refer to ac- tions of pouring, inserting or laying a body part. In the Example 2 there is an action of beating up someone (Fig- ure 1), that can be correctly encoded by two senses of the verb to beat. Example 1 bn:00090224v1: place, lay, put, set, pose. Put into a cer- tain place or abstract location. 1The IDs reported in Example 1 and 2 are taken from BabelNet taxonomy, that derives directly from WordNet. (see 2.2. for more details. • John puts the wine in the glass; • John puts the letter in …

Content truncated to 500 words