Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 108–117, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction Lanbo She and Joyce Y. Chai Department of Computer Science and Engineering Michigan State University East Lansing, Michigan 48824, USA {shelanbo, jchai}@cse.msu.edu Abstract As a new generation of cognitive robots start to enter our lives, it is important to enable robots to follow human commands and to learn new actions from human lan- guage instructions. To address this issue, this paper presents an approach that ex- plicitly represents verb semantics through hypothesis spaces of fluents and automat- ically acquires these hypothesis spaces by interacting with humans. The learned hy- pothesis spaces can be used to automati- cally plan for lower-level primitive actions towards physical world interaction. Our empirical results have shown that the rep- resentation of a hypothesis space of flu- ents, combined with the learned hypothe- sis selection algorithm, outperforms a pre- vious baseline. In addition, our approach applies incremental learning, which can contribute to life-long learning from hu- mans in the future. 1 Introduction As a new generation of cognitive robots start to enter our lives, it is important to enable robots to follow human commands (Tellex et al., 2014; Thomason et al., 2015) and to learn new actions from human language instructions (Cantrell et al., 2012; Mohan et al., 2013). To achieve such a capability, one of the fundamental challenges is to link higher-level concepts expressed by human language to lower-level primitive actions the robot is familiar with. While grounding language to perception (Gorniak and Roy, 2007; Chen and Mooney, 2011; Kim and Mooney, 2012; Artzi and Zettlemoyer, 2013; Tellex et al., 2014; Liu et al., 2014; Liu and Chai, 2015) has received much at- tention in recent years, less work has addressed grounding language to robotic action. Actions are often expressed by verbs or verb phrases. Most semantic representations for verbs are based on ar- gument frames (e.g., thematic roles which capture participants of an action). For example, suppose a human directs a robot to “fill the cup with milk”. The robot will need to first create a semantic rep- resentation for the verb “fill” where “the cup” and “milk” are grounded to the respective objects in the environment (Yang et al., 2016). Suppose the robot is successful in this first step, it still may not be able to execute the action “fill” as it does not know how this higher-level action corresponds to its lower-level primitive actions. In robotic systems, operations usually consist of multiple segments of lower-level primitive actions (e.g., move to, open gripper, and close gripper) which are executed both sequentially and con- currently. Task scheduling provides the order or schedule for executions of different segments of actions and action planning provides the plan for executing each individual segment. Primitive ac- tions are often predefined in terms of how they change the state of the physical world. Given a …

Content truncated to 500 words