CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15339–15353 July 9-14, 2023 ©2023 Association for Computational Linguistics CODEIE: Large Code Generation Models are Better Few-Shot Information Extractors Peng Li1,∗ Tianxiang Sun2,∗ Qiong Tang2 Hang Yan2 Yuanbin Wu3 Xuanjing Huang2 Xipeng Qiu2,† 1Academy for Engineering & Technology, Fudan University 2School of Computer Science, Fudan University 3School of Computer Science and Technology, East China Normal University {lip21,qtang22}@m.fudan.edu.cn, ybwu@cs.ecnu.edu.cn {txsun19,hyan19,xjhuang,xpqiu}@fudan.edu.cn Abstract Large language models (LLMs) pre-trained on massive corpora have demonstrated impressive few-shot learning ability on many NLP tasks. A common practice is to recast the task into a text-to-text format such that generative LLMs of natural language (NL-LLMs) like GPT-3 can be prompted to solve it. However, it is non- trivial to perform information extraction (IE) tasks with NL-LLMs since the output of the IE task is usually structured and therefore is hard to be converted into plain text. In this paper, we propose to recast the structured output in the form of code instead of natural language and utilize generative LLMs of code (Code-LLMs) such as Codex to perform IE tasks, in particu- lar, named entity recognition and relation ex- traction. In contrast to NL-LLMs, we show that Code-LLMs can be well-aligned with these IE tasks by designing code-style prompts and formulating these IE tasks as code generation tasks. Experiment results on seven benchmarks show that our method consistently outperforms fine-tuning moderate-size pre-trained models specially designed for IE tasks (e.g., UIE) and prompting NL-LLMs under few-shot settings. We further conduct a series of in-depth anal- yses to demonstrate the merits of leveraging Code-LLMs for IE tasks.1 1 Introduction Information extraction (IE) aims to recognize struc- tured information from plain text. It spans various tasks with diverse output structures such as named entity recognition (NER), relation extraction (RE), etc. (Sang and Meulder, 2003; Grishman, 2019; Wang et al., 2021a; Zhong and Chen, 2021; Lu et al., 2022). To express and address these different tasks in a unified framework, recent works propose to linearize the output structures into unstructured ∗ Equal contribution. † Corresponding author. 1Code is available at https://github.com/dasepli/CodeIE industrial designer, and CEO of Apple; NL-LLMs Steven Paul Jobs was an Pre-train The input text is "…". The entities … (person: …) (organization: …) Inference (a) Performing NER with NL-LLMs Code-LLMs list1 = [1, 2, 3] list2 = [4, 5, 6, 7] list1.append(…) list2.append(…) Pre-train input_text = "…" entity_list = [] entity_list.append(…) entity_list.append(…) Inference (b) Performing NER with Code-LLMs Figure 1: Illustrations of performing structured NER task with NL-LLMs and Code-LLMs, respectively. In contrast to prompting NL-LLMs with plain natural lan- guage, we utilize Code-LLMs with structured code-style prompts to mitigate the output discrepancy between the pre-training and inference stages. strings and solve the IE tasks with sequence gen- eration models (Yan et al., 2021b; Huguet Cabot and Navigli, 2021; Paolini et al., 2021; Josifoski et al., 2022; Lu et al., 2022). For example, given the input sentence "Steve became CEO of Apple …

Content truncated to 500 words