A Two-Level Interpretation of Modality in Human-Robot Dialogue

Proceedings of the 28th International Conference on Computational Linguistics, pages 4222–4238 Barcelona, Spain (Online), December 8-13, 2020 4222 A Two-Level Interpretation of Modality in Human-Robot Dialogue Lucia Donatelli1, Kenneth Lai2, James Pustejovsky2 1Department of Language Science and Technology Saarland University, Germany 2Department of Computer Science Brandeis University, USA donatelli@coli.uni-saarland.de {klai12,jamesp}@brandeis.edu Abstract We analyze the use and interpretation of modal expressions in a corpus of situated human-robot dialogue and ask how to effectively represent these expressions for automatic learning and dynamic interpretation in context. We present a two-level annotation scheme for modality that captures both content and intent, integrating a logic-based, semantic representation and a task-oriented, pragmatic representation that maps to our robot’s capabilities. Data from our annotation task reveals that the interpretation of modal expressions in human-robot dialogue is quite diverse, yet highly constrained by the physical environment and asymmetrical speaker/addressee relationship. We sketch a formal model of human-robot common ground in which modality can be grounded and dynamically interpreted relative to speaker role, temporal constraints, and physical environment. 1 Introduction The interpretation of modal expressions is essential to meaningful human-robot dialogue: the ability to convey information about objects and events that are displaced in time, space, and actuality allows the human and robot to align their environmental perceptions and successfully collaborate (Liu and Chai, 2015). As an example, if a robot is sent to a remote location on a search and navigation mission, modally interpreted expressions such as “Tell me what you see” (uttered by the human) and “I can’t see because of smoke” (uttered by the robot) are vital to information exchange. Similarly, a robot that has abilities to navigate obstacles (for example, by jumping or using LIDAR) can inform the human of this. The learning of modal expressions for automatic understanding and use nevertheless presents a con- versational paradox: while these expressions serve to communicate and align world knowledge, there is no obvious manner to ground them in the shared environment. Whereas objects and actions can be pointed to or modeled for grounded learning, modal expressions are grounded in the linguistic signal itself. Nevertheless, a basic understanding of modal meaning would allow non-human agents to reason about the possible uses of objects and better assess how certain actions and behaviors impact the task at hand. In this paper, we document the range and nature of modally interpreted expressions used in human-robot dialogue with the goal to make the interpretation of such expressions easily automated in the future. We hypothesize that certain readings and scope preferences for modal operators are more salient in human-robot dialogue because of the unique makeup of the common ground (Poesio, 1993). We provide a mapping from formal semantic theories of modality related to participant beliefs and updates of the common ground (Portner, 2009), to a practical model of speech acts that translates into robot action for search and navigation task-oriented dialogue and an automated NLU and NLG system (Bonial et al., 2020). This mapping is formalized in an annotation scheme in which the …

Content truncated to 500 words