DWUG: A large Resource of Diachronic Word Usage Graphs in Four Languages

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 734–741 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 734 Injecting Entity Types into Entity-Guided Text Generation Xiangyu Dong1∗, Wenhao Yu1∗, Chenguang Zhu2, Meng Jiang1 1University of Notre Dame, Notre Dame, IN, 2Microsoft Cognitive Services Research, Redmond, WA xdong2ps@gmail.com, wyu1@nd.edu chezhu@microsoft.com, mjiang2@nd.edu Abstract Recent successes in deep generative modeling have led to significant advances in natural lan- guage generation (NLG). Incorporating enti- ties into neural generation models has demon- strated great improvements by assisting to in- fer the summary topic and to generate coherent content. To enhance the role of entity in NLG, in this paper, we aim to model the entity type in the decoding phase to generate contextual words accurately. We develop a novel NLG model to produce a target sequence based on a given list of entities. Our model has a multi- step decoder that injects the entity types into the process of entity mention generation. Ex- periments on two public news datasets demon- strate type injection performs better than exist- ing type embedding concatenation baselines. 1 Introduction Entity, as an important element of natural lan- guage, plays the key role of making the text co- herent (Grosz et al., 1995). Recently, modeling entities into NLG methods has demonstrated great improvements by assisting to infer the summary topic (Amplayo et al., 2018) or to generate coher- ent content (Ji et al., 2017; Clark et al., 2018). To enhance the representation of entity, entity type is often used in existing work – represented as a separate embedding and concatenated with the em- bedding of entity mention (i.e., surface name) in the encoding/decoding phase (Zhao et al., 2019; Puduppully et al., 2019; Yu et al., 2018; Chan et al., 2019). Although the concatenation performed bet- ter than using the entity mention embedding only, the relationship between entity mention and entity type was not reflected, making the signal from en- tity type undermined in the NLG. To address the above issue, our idea is to model the entity type carefully in the decoding phase to * The first two authors have equal contributions. § Our code and datasets are available at https:// github.com/DM2-ND/InjType. Table 1: An example of generating news with a list of names of entities and their types. How to use entity type information in NLG models is an open question. Input: [COUNTRY:US1, PERSON:Dick_Cheney2, COUNTRY:Afghanistan3, WEEKDAY:Monday4, COUNTRY:Afghan5, PERSON:Hamid_Karzai6, ORGANIZATION:NATO7, CITY:Bucharest8] Target: “US1 vice president Dick_Cheney2 made a surprise visit to Afghanistan3 on Monday4 for talks with Afghan5 president Hamid_Karzai6, ahead of the NATO7 summit early next month in Bucharest8.” generate contextual words accurately. In this work, we focus on developing a novel NLG model to pro- duce a target sequence based on a given list of enti- ties. Compared to the number of words in the tar- get sequence, the number of given entities is much smaller. Since the source information is extremely insufficient, it is difficult to generate precise con- textual words describing the …

Content truncated to 500 words