Generative Entity Typing with Curriculum Learning
31284Authors
nan
Abstract
AbstractEntity typing aims to assign types to the entity mentions in given texts. The traditional classification-based entity typing paradigm has two unignorable drawbacks: 1) it fails to assign an entity to the types beyond the predefined type set, and 2) it can hardly handle few-shot and zero-shot situations where many long-tail types only have few or even no training instances. To overcome these drawbacks, we propose a novel generative entity typing (GET) paradigm: given a text with an entity mention, the multiple types for the role that the entity plays in the text are generated with a pre-trained language model (PLM). However, PLMs tend to generate coarse-grained types after fine-tuning upon the entity typing dataset. In addition, only the heterogeneous training data consisting of a small portion of human-annotated data and a large portion of auto-generated but low-quality data are provided for model training. To tackle these problems, we employ curriculum learning (CL) to train our GET model on heterogeneous data, where the curriculum could be self-adjusted with the self-paced learning according to its comprehension of the type granularity and data heterogeneity. Our extensive experiments upon the datasets of different languages and downstream tasks justify the superiority of our GET model over the state-of-the-art entity typing models. The code has been released on https://github.com/siyuyuan/GET.
Keywords
Full Content
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 277 - 290 December 7-11, 2022 ©2022 Association for Computational Linguistics Multi-VQG: Generating Engaging Questions for Multiple Images Min-Hsuan Yeh∗, Vicent Chen⋄, Ting-Hao ‘Kenneth’ Haung†, Lun-Wei Ku‡ University of Massachusetts Amherst∗, University of Illinois Urbana-Champaign⋄, Pennsylvania State University†, Institute of Information Science, Academia Sinica‡ myeh@umass.edu, vfchen2@illinois.edu, txh710@psu.edu, lwku@iis.sinica.edu Abstract Generating engaging content has drawn much recent attention in the NLP community. Ask- ing questions is a natural way to respond to photos and promote awareness. However, most answers to questions in traditional question- answering (QA) datasets are factoids, which reduce individuals’ willingness to answer. Fur- thermore, traditional visual question genera- tion (VQG) confines the source data for ques- tion generation to single images, resulting in a limited ability to comprehend time-series in- formation of the underlying event. In this pa- per, we propose generating engaging questions from multiple images. We present MVQG1, a new dataset, and establish a series of baselines, including both end-to-end and dual-stage ar- chitectures. Results show that building stories behind the image sequence enables models to generate engaging questions, which confirms our assumption that people typically construct a picture of the event in their minds before ask- ing questions. These results open up an excit- ing challenge for visual-and-language models to implicitly construct a story behind a series of photos to allow for creativity and experi- ence sharing and hence draw attention to down- stream applications. 1 Introduction The popularity of image-sharing behavior in chats and social media applications shows that this is a natural way to increase participant engagement (Hu et al., 2014). In response, asking questions based on these photos is a straightforward method to pro- mote awareness, sustain attention, and acquire use- ful information. An obvious example is that when we see someone share a photo of a car accident on Facebook, commenting “Was anyone injured in the crash?” draws more attention and replies from both the author and other readers than “Oh 1Github repo: https://github.com/AcademiaSinicaNLPLab /MVQG-Dataset-of-Generating-Engaging-Questions-for-Mu ltiple-Images How would you act if you found yourself in a room filled with cans of free drinks? Have you ever gone to beer tastings and where would that be at? How long did the cat lounge around in the book room? What would this cat sit on next? Figure 1: Two examples of MVQG. Each data point consists of an image sequence and two to five engaging questions written by humans. The machine should gen- erate a question over a given image sequence. my goodness, that is serious.” Furthermore, from the author’s aspect, posting photos on social media with an engaging, image-related question helps the author to hear the public voice of their feelings and thoughts, and keeps the author connected with the world (Lu et al., 2021). However, not all the questions have the same effect. Questions in traditional text-based QA datasets such as SQuAD (Rajpurkar et al., 2016), NarrativeQA (Kociský et al., 2018), or Fairy- taleQA (Zhao et al., 2022) …
Content truncated to 500 words
Labeled Text
AbstractEntity typing aims to assign types to the entity mentions in given texts. The traditional classification-based entity typing paradigm has two unignorable drawbacks: 1) it fails to assign an entity to the types beyond the predefined type set, and 2) it can hardly handle few-shot and zero-shot situations where many long-tail types only have few or even no training instances. To overcome these drawbacks, we propose a novel generative entity typing (GET) paradigm: given a text with an entity mention, the multiple types for the role that the entity plays in the text are generated with a pre-trained language model (PLM). However, PLMs tend to generate coarse-grained types after fine-tuning upon the entity typing dataset. In addition, only the heterogeneous training data consisting of a small portion of human-annotated data and a large portion of auto-generated but low-quality data are provided for model training. To tackle these problems, we employ curriculum learning (CL) to train our GET model on heterogeneous data, where the curriculum could be self-adjusted with the self-paced learning according to its comprehension of the type granularity and data heterogeneity. Our extensive experiments upon the datasets of different languages and downstream tasks justify the superiority of our GET model over the state-of-the-art entity typing models. The code has been released on
Publication Details
- Paper ID:
- 31284
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan