Linguistic representations for fewer-shot relation extraction across domains

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7502–7514 July 9-14, 2023 ©2023 Association for Computational Linguistics Linguistic representations for fewer-shot relation extraction across domains Sireesh Gururaja and Ritam Dutt and Tinglong Liao and Carolyn Rosé Language Technologies Institute Carnegie Mellon University {sgururaj, rdutt, tinglonl, cprose}@cs.cmu.edu Abstract Recent work has demonstrated the positive im- pact of incorporating linguistic representations as additional context and scaffolding on the in-domain performance of several NLP tasks. We extend this work by exploring the impact of linguistic representations on cross-domain performance in a few-shot transfer setting. An important question is whether linguistic rep- resentations enhance generalizability by pro- viding features that function as cross-domain pivots. We focus on the task of relation ex- traction on three datasets of procedural text in two domains, cooking and materials science. Our approach augments a popular transformer- based architecture by alternately incorporating syntactic and semantic graphs constructed by freely available off-the-shelf tools. We examine their utility for enhancing generalization, and investigate whether earlier findings, e.g. that semantic representations can be more helpful than syntactic ones, extend to relation extrac- tion in multiple domains. We find that while the inclusion of these graphs results in signifi- cantly higher performance in few-shot transfer, both types of graph exhibit roughly equivalent utility. 1 Introduction In many specialized domains, such as healthcare or finance, one of the principal limitations for the im- plementation of machine learning based NLP meth- ods is the availability of high quality data, which tends to be both time-consuming and expensive to acquire. While pre-trained language models allow impressive performance gains across a number of tasks, those gains frequently fail to generalize to specialized domains. Robust representations that allow models to both take advantage of pretrained models and generalize across domains are therefore highly desirable. Recent works such as Prange et al. (2022) have demonstrated the significant potential of using human-annotated linguistic information as scaffold- ing for learning language models. Other works such as Zhang and Ji (2021) and Bai et al. (2021) use automatically generated semantic annotations. These works depend on the idea that the structure that the linguistic frameworks provide allows mod- els to better learn salient features of the input. In ad- dition, however, linguistic structures offer abstrac- tion over the variation of natural language, provid- ing representations that might express meaning in domain-general ways. We therefore extend earlier work to investigate whether including linguistic rep- resentations encourages learning domain-agnostic representations of relations such that models can generalize better in a few-shot setting, i.e. learn- ing from less high-quality data in new domains. We focus on the case of automatically generated linguistic annotations, to evaluate the impact they can have on downstream tasks without expensive human annotation of parse data. We use two linguistic formalisms, to evaluate and compare their impact: dependency parses, and abstract meaning representations (AMR). AMR (Banarescu et al., 2013) seeks to represent meaning at the level of a sentence in the form of …

Content truncated to 500 words