Title2Event: Benchmarking Open Event Extraction with a Large-scale Chinese Title Dataset

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 689 - 705 December 7-11, 2022 ©2022 Association for Computational Linguistics A Unified Encoder-Decoder Framework with Entity Memory Zhihan Zhang1, Wenhao Yu1, Chenguang Zhu2, Meng Jiang1 1University of Notre Dame, Notre Dame, IN, USA 2Microsoft Cognitive Services Research, Redmond, WA, USA 1{zzhang23, wyu1, mjiang2}@nd.edu; 2chezhu@microsoft.com Abstract Entities, as important carriers of real-world knowledge, play a key role in many NLP tasks. We focus on incorporating entity knowledge into an encoder-decoder framework for infor- mative text generation. Existing approaches tried to index, retrieve, and read external doc- uments as evidence, but they suffered from a large computational overhead. In this work, we propose an Encoder-Decoder framework with an entity Memory, namely EDMem. The entity knowledge is stored in the memory as latent representations, and the memory is pre-trained on Wikipedia along with encoder-decoder pa- rameters. To precisely generate entity names, we design three decoding methods to constrain entity generation by linking entities in the mem- ory. EDMem is a unified framework that can be used on various entity-intensive question an- swering and generation tasks. Extensive exper- imental results show that EDMem outperforms both memory-based auto-encoder models and non-memory encoder-decoder models. 1 Introduction A large amount of real-world knowledge is related to entities, e.g., persons, nations, and events. Entity knowledge is the information describing facts and attributes related to entities. Many entity-intensive NLP tasks require models obtain entity knowledge to generate informative outputs, such as answer- ing factual questions (Kwiatkowski et al., 2019), explaining claims (Onoe et al., 2021), or making informative conversations (Dinan et al., 2019). Pre- trained encoder-decoder models can be directly ap- plied on such entity-intensive tasks (Ye et al., 2020; Roberts et al., 2020), but their ability to store and use knowledge is still questionable (Lewis et al., 2021; Wang et al., 2021). A popular approach to incorporate knowledge into the generation pro- cess is retrieving evidence documents from external sources (Lewis et al., 2020b; Izacard and Grave, Transformer Transformer Who wrote [Es] Evening Class [Ee] ? + Entity Linking Head Entity Memory Transformer Transformer [BOS] [Es] Maeve Binchy + [Es] Maeve Binchy [Ee] Language Modeling Head Encoder Decoder Figure 1: An overview of the EDMem framework. H denotes the final hidden states of the encoder. 2021; Oguz et al., 2020; Yu et al., 2022c). How- ever, they suffer from significant computational overheads in indexing, retrieving, and reading a large number of extra documents (Lee et al., 2021; de Jong et al., 2022). Therefore, it is important to give encoder-decoder models access to entity knowledge without sacrificing too much efficiency. Recently it has been proposed to use an in-model memory to augment auto-encoder models with en- tity knowledge on entity linking tasks (Févry et al., 2020; Verga et al., 2021; Sun et al., 2021). The en- tity memory stores entity knowledge as dense vec- tors which can be directly incorporated into the hid- den states of Transformer models (Vaswani et al., 2017), with no need …

Content truncated to 500 words