From Spatial Relations to Spatial Configurations

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5855–5864 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5855 From Spatial Relations to Spatial Configurations Soham Dan 1, Parisa Kordjamshidi3,4, Julia Bonn2, Archna Bhatia3, Jon Cai2, Martha Palmer2, Dan Roth1 1University of Pennsylvania, 2University of Colorado, Bolder, 3IHMC, 4Michigan State University {sohamdan,danroth}@seas.upenn.edu , {julia.bonn,jon.z.cai,martha.palmer}@colorado.edu , kordjams@msu.edu, abhatia@ihmc.us Abstract Spatial Reasoning from language is essential for natural language understanding. Supporting it requires a representation scheme that can capture spatial phenomena encountered in language as well as in images and videos. Existing spatial representations are not sufficient for describing spatial configurations used in complex tasks. This paper extends the capabilities of existing spatial representation languages and increases coverage of the semantic aspects that are needed to ground spatial meaning of natural language text in the world. Our spatial relation language is able to represent a large, comprehensive set of spatial concepts crucial for reasoning and is designed to support composition of static and dynamic spatial configurations. We integrate this language with the Abstract Meaning Representation (AMR) annotation schema and present a corpus annotated by this extended AMR. To exhibit the applicability of our representation scheme, we annotate text taken from diverse datasets and show how we extend the capabilities of existing spatial representation languages with fine-grained decomposition of semantics and blend it seamlessly with AMRs of sentences and discourse representations as a whole. Keywords: Semantics, Knowledge Discovery/Representation 1. Introduction Spatial reasoning is necessary for many tasks. For example, consider the collaborative building task (Narayan-Chen et al., 2019) where a human architect (knows the target structure but cannot manipulate blocks) issues commands to a builder (who does not know the target structure but can place and manipulate blocks) with the aim of creat- ing a complex target structure in a grounded environment (Minecraft). This task is challenging from a spatial reason- ing perspective because the architect does not issue com- mands in terms of the unit blocks that the builder uses, but complex shapes and their spatial aspects. A single instruc- tion can convey information about complex spatial configu- rations of several objects and describe nested relationships while shifting focus from describing one object to another. For example, in Place the red block on the yellow block which is to the left of the blue column., the focus shifts from describing the intended location of the red block to describ- ing the location of the yellow block. Or, in a navigation description, there can be a sequence of spatial expressions describing a single object, as in Over the hill and through the woods.... There is a severe limitation in the ability of ex- isting spatial representation schemes to represent sentences involving complex spatial concepts. An example sentence from the Minecraft corpus is shown in Figure 1. Table 1 highlights some of the challenging instances from a few recent datasets that cannot be adequately represented by existing spatial representations( for a more detailed com- parison …

Content truncated to 500 words