Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6148 - 6165 December 7-11, 2022 ©2022 Association for Computational Linguistics Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning Roshanak Mirzaee Michigan State University mirzaee@msu.edu Parisa Kordjamshidi Michigan State University kordjams@msu.edu Abstract Recent research shows synthetic data as a source of supervision helps pretrained language models (PLM) transfer learning to new tar- get tasks/domains. However, this idea is less explored for spatial language. We provide two new data resources on multiple spatial language processing tasks. The first dataset is synthesized for transfer learning on spatial question answering (SQA) and spatial role la- beling (SpRL). Compared to previous SQA datasets, we include a larger variety of spatial relation types and spatial expressions. Our data generation process is easily extendable with new spatial expression lexicons. The second one is a real-world SQA dataset with human- generated questions built on an existing corpus with SPRL annotations. This dataset can be used to evaluate spatial language processing models in realistic situations. We show pre- training with automatically generated data sig- nificantly improves the SOTA results on several SQA and SPRL benchmarks, particularly when the training data in the target domain is small. 1 Introduction Understanding spatial language is important in many applications such as navigation (Zhang and Kordjamshidi, 2022; Zhang et al., 2021; Chen et al., 2019), medical domain (Datta et al., 2020; Kamel Boulos et al., 2019; Massa et al., 2015), and robotics (Venkatesh et al., 2021; Kennedy et al., 2007). However, few benchmarks have directly focused on comprehending the spatial semantics of the text. Moreover, the existing datasets are either synthetic (Mirzaee et al., 2021; Weston et al., 2015; Shi et al., 2022) or at small scale (Mirzaee et al., 2021; Kordjamshidi et al., 2017). The synthetic datasets often focus on specific types of relations with a small coverage of spa- tial semantics needed for spatial language under- standing in various domains. Figure 2 indicates the coverage of sixteen spatial relation types (in Q: Is the yellow apple to the west of the yellow watermelon? Yes Three boxes called one, two and three exist in an image. Box one contains a big yellow melon and a small orange watermelon. Box two has a small yellow apple. A small orange apple is inside and touching this box. Box one is in box three. Box two is to the south of, far from and to the west of box three. A small yellow watermelon is inside box three. Q: Where is box two relative to the yellow watermelon? Left, Below, Far (a) SPARTUN - A synthetic large dataset provided as a source of supervision. A grey car is parking in front of a grey house with brown window frames and plants on the balcony. Q: Are the plants in front of the car? No Q: Are the plants in the house? Yes (b) RESQ - A human-generated dataset for probing the models on realistic SQA Figure 1: Two …

Content truncated to 500 words