Code4Struct: Code Generation for Few-Shot Event Structure Prediction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3640–3663 July 9-14, 2023 ©2023 Association for Computational Linguistics CODE4STRUCT: Code Generation for Few-Shot Event Structure Prediction Xingyao Wang and Sha Li and Heng Ji University of Illinois Urbana-Champaign, IL, USA {xingyao6, shal2, hengji}@illinois.edu Abstract Large Language Model (LLM) trained on a mixture of text and code has demon- strated impressive capability in translating natural language (NL) into structured code. We observe that semantic structures can be conveniently translated into code and pro- pose CODE4STRUCT to leverage such text-to- structure translation capability to tackle struc- tured prediction tasks. As a case study, we for- mulate Event Argument Extraction (EAE) as converting text into event-argument structures that can be represented as a class object using code. This alignment between structures and code enables us to take advantage of Program- ming Language (PL) features such as inheri- tance1 and type annotation2 to introduce exter- nal knowledge or add constraints. We show that, with sufficient in-context examples, for- mulating EAE as a code generation problem is advantageous over using variants of text-based prompts. Despite only using 20 training event instances for each event type, CODE4STRUCT is comparable to supervised models trained on 4,202 instances and outperforms current state- of-the-art (SOTA) trained on 20-shot data by 29.5% absolute F1. By leveraging the inheri- tance feature of PL, CODE4STRUCT can use 10-shot training data from a sibling event type to predict arguments for zero-resource event types and outperforms the zero-shot baseline by 12% absolute F1. 3 1 Introduction Large Language Model (LLM) trained on massive corpora of code mixed with natural language (NL) comments and docstrings4 (e.g., Chen et al. 2021, 1Inheritance is a way to create a hierarchy of classes in PL. A child class can base upon another class, retaining similar implementation. 2Developers use type annotations to indicate the data types of variables and input/outputs of functions. 3All code and resources are publicly available at https: //github.com/xingyaoww/code4struct. 4Text used to document a specific segment of code. class Transport(Movement): ... Transport (Event Type) GPE or ORG or PER FAC or ORG or PER or VEH or WEA LAC or GPE or LOC FAC or GPE or LOC VEH agent artifact destination origin vehicle (1) Event Ontology (2) Event Definition """ Translate the following sentence into an instance of Transport. The trigger word(s) of the event is marked with **trigger word**. "Kelly , the US assistant secretary for East Asia and Pacific Affairs , **arrived** in Seoul from Beijing Friday to brief Yoon , the foreign minister ." """ transport_event = Transport( artifact=[ PER("Kelly"), ], destination=[ GPE("Seoul"), ], origin=[ GPE("Beijing"), ], ) Input Sentence Generated Code (3) Event Instantiation Convert to Python class Transport (Event Instance) PER: Kelly GPE: Seoul GPE: Beijing agent destination origin Prompt LLM Figure 1: Event Argument Extraction using code gen- eration. We convert the existing event type ontology to PYTHON class definitions. Conditioned on these defi- nitions, we put the input sentence for …

Content truncated to 500 words