Generative Model

It is challenging to translate names and technical terms across languages with differ- ent alphabets and sound inventories. These items are commonly transliterated, i.e., re- placed with approximate phonetic equivalents. …...

The model used by the CCG parser of Hockenmaier and Steedman (2002b) would fail to capture the correct bilexical dependencies in a language with freer word order, such as Dutch. …...

We present a generative model for the unsupervised learning of dependency structures. We also describe the multiplicative combination of this dependency model with a model of linear constituency. The product …...

This paper describes an incremental parsing approach where parameters are estimated using a variant of the perceptron algorithm. A beam-search algorithm is used during both training and decoding phases of …...

A given entity, representing a person, a location or an organization, may be mentioned in text in multiple, ambiguous ways. Understanding natural language requires identifying whether different mentions of a …...

In this paper we show that generative models are competitive with and some- times superior to discriminative models, when both kinds of models are allowed to learn structures that are …...

Unsupervised learning of linguistic structure is a difficult problem. A common approach is to define a generative model and max- imize the probability of the hidden struc- ture given the …...

In this paper, we present an algorithm for learning a generative model of natural lan- guage sentences together with their for- mal meaning representations with hierarchi- cal structures. The model …...

This paper describes a computational ap- proach to resolving the true referent of a named mention of a person in the body of an email. A generative model of mention …...

We present a generative model for unsuper- vised coreference resolution that views coref- erence as an EM clustering process. For comparison purposes, we revisit Haghighi and Klein’s (2007) fully-generative Bayesian …...

Spoken Language Understanding aims at mapping a natural language spoken sen- tence into a semantic representation. In the last decade two main approaches have been pursued: native models.generative and discrimi- …...

s Jacob Eisenstein∗ James Clarke† Dan Goldwasser† Dan Roth∗† ∗Beckman Institute for Advanced Science and Technology, †Department of Computer Science University of Illinois Urbana, IL 61801 {jacobe,clarkeje,goldwas1,danr}@illinois.edu Abstract Machine learning …...

We describe a generative model for clustering named entities which also models named en- tity internal structure, clustering related words by role. The model is entirely unsupervised; it uses features …...

The character-based tagging approach is a dominant technique for Chinese word segmentation, and both discrimi- native and generative models can be adopted in that framework. However, generative and discriminative charac- …...

Scoring sentences in documents given ab- stract summaries created by humans is im- portant in extractive multi-document sum- marization. In this paper, we formulate ex- tractive summarization as a two …...

The rapid growth of geotagged social media raises new computational possibilities for in- vestigating geographic linguistic variation. In this paper, we present a multi-level generative model that reasons jointly about …...

We show how features can easily be added to standard generative models for unsuper- vised learning, without requiring complex new training methods. In particular, each component multinomial of a generative …...

AbstractIn the POS tagging task, there are two kinds of statistical models: one is generative model, such as the HMM, the others are discriminative models, such as the Maximum Entropy …...

This paper presents a generative model for the automatic discovery of relations between entities in electronic medical records. The model discovers relation instances and their types by determining which context …...

This paper describes a novel probabilistic ap- proach for generating natural language sen- tences from their underlying semantics in the form of typed lambda calculus. The approach is built on …...

We consider a semi-supervised setting for do- main adaptation where only unlabeled data is available for the target domain. One way to tackle this problem is to train a generative …...

We describe a generative model for non- projective dependency parsing based on a sim- plified version of a transition system that has recently appeared in the literature. We then develop …...

This paper introduces tree transducers as a unifying theory for semantic parsing models based on tree transformations. Many exist- ing models use tree transformations, but im- plement specialized training and …...

We address the task of automatic discovery of information extraction template from a given text collection. Our approach clusters candi- date slot fillers to identify meaningful tem- plate slots. We …...

The emergence of dialogue on social medial neccessitates the development of new dialogue processing models. We argue that to address coherence and to infer the implicatures of social dialogue it …...

Features for Robust Cross-Domain Chinese Word Segmentation LI Xiaoqing1, WANG Kun1, ZONG Chengqing1 and SU Keh-Yih2 (1) National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, …...

Humans identify word boundaries in continu- ous speech by combining multiple cues; exist- ing state-of-the-art models, though, look at a single cue. We extend the generative model of Goldwater et …...

Event schema induction is the task of learning high-level representations of complex events (e.g., a bombing) and their entity roles (e.g., perpetrator and victim) from unlabeled text. Event schemas have …...

In this paper, we propose a new method for semantic class induction. First, we in- troduce a generative model of sentences, based on dependency trees and which takes into account …...

AbstractDependency cohesion refers to the observation that phrases dominated by disjoint dependency subtrees in the source language generally do not overlap in the target language. It has been verified to …...

Most of the machine translation systems rely on a large set of translation rules. These rules are treated as discrete and independent events. In this short paper, we propose a …...

AbstractCombining Translation Memory (TM) with Statistical Machine Translation (SMT) together has been demonstrated to be beneficial. In this paper, we present a discriminative framework which can integrate TM into SMT …...

We propose the use of a generative model to simulate user behaviour in a novel task- oriented dialog domain, where user goals are spatial routes across artificial land- scapes. We …...

In this article, we describe a new approach to distributional semantics. This approach relies on a generative model of sentences with latent variables, which takes the syntax into account by …...

State-of-the-art Chinese zero pronoun res- olution systems are supervised, thus re- lying on training data containing manu- ally resolved zero pronouns. To elimi- nate the reliance on annotated data, we …...

AbstractAs time passes words can acquire meanings they did not previously have, such as the ‘twitter post’ usage of ‘tweet’. We address how this can be detected from time-stamped raw …...

We introduce recurrent neural network gram- mars, probabilistic models of sentences with explicit phrase structure. We explain efficient inference procedures that allow application to both parsing and language modeling. Experi- …...

We present a novel semi-supervised approach for sequence transduction and apply it to se- mantic parsing. The unsupervised component is based on a generative model in which latent sentences generate …...

AbstractWe present a generative model of natural language sentences and demonstrate its application to semantic parsing. In the generative process, a logical form sampled from a prior, and conditioned on …...

AbstractWe apply adversarial training to open-domain dialogue generation, training a system to produce sequences that are indistinguishable from human-generated dialogue utterances. We cast the task as a reinforcement learning problem …...

AbstractLexical resources such as dictionaries and gazetteers are often used as auxiliary data for tasks such as part-of-speech induction and named-entity recognition. However, discriminative training with lexical features requires annotated …...

AbstractA document outlier is a document that substantially deviates in semantics from the majority ones in a corpus. Automatic identification of document outliers can be valuable in many applications, such …...

AbstractTreebanks traditionally treat punctuation marks as ordinary words, but linguists have suggested that a tree’s “true” punctuation marks are not observed (Nunberg, 1990). These latent “underlying” marks serve to delimit …...

AbstractWe propose a generative model for a sentence that uses two latent variables, with one intended to represent the syntax of the sentence and the other to represent its semantics. …...

AbstractIn the principles-and-parameters framework, the structural features of languages depend on parameters that may be toggled on or off, with a single parameter often dictating the status of multiple features. …...

AbstractIn this paper, we present a novel integrated approach for keyphrase generation (KG). Unlike previous works which are purely extractive or generative, we first propose a new multi-task learning framework …...

AbstractCross-lingual transfer is an effective way to build syntactic analysis tools in low-resource languages. However, transfer is difficult when transferring to typologically distant languages, especially when neither annotated target data …...

AbstractLearning to hash via generative model has become a powerful paradigm for fast similarity search in documents retrieval. To get binary representation (i.e., hash codes), the discrete distribution prior (i.e., …...

AbstractShort texts such as tweets often contain insufficient word co-occurrence information for training conventional topic models. To deal with the insufficiency, we propose a generative model that aggregates short texts …...

AbstractNatural language understanding (NLU) and natural language generation (NLG) are two fundamental and related tasks in building task-oriented dialogue systems with opposite objectives: NLU tackles the transformation from natural language …...

AbstractWe propose a novel generative model to explore both local and global context for joint learning topics and topic-specific word embeddings. In particular, we assume that global latent topics are …...

AbstractWe describe a weakly-supervised method for training deep learning models for the task of ad-hoc document retrieval. Our method is based on generative and discriminative models that are trained using …...

AbstractVisualization and topic modeling are widely used approaches for text analysis. Traditional visualization methods find low-dimensional representations of documents in the visualization space (typically 2D or 3D) that can be …...

AbstractOpinion summarization is the task of automatically creating summaries that reflect subjective information expressed in multiple documents, such as product reviews. While the majority of previous work has focused on …...

AbstractIn this paper, we tackle the task of definition modeling, where the goal is to learn to generate definitions of words and phrases. Existing approaches for this task are discriminative, …...

AbstractIn the pursuit of natural language understanding, there has been a long standing interest in tracking state changes throughout narratives. Impressive progress has been made in modeling the state of …...

AbstractNamed entity recognition (NER) is a fundamental task to recognize specific types of entities from a given sentence. Depending on how the entities appear in the sentence, it can be …...

AbstractMost previous methods for text data augmentation are limited to simple tasks and weak baselines. We explore data augmentation on hard tasks (i.e., few-shot natural language understanding) and strong baselines …...

AbstractThis paper presents the second place system for the R2VQ: competence-based multimodal question answering shared task. The purpose of this task is to involve semantic&cooking roles and text-images objects when …...

AbstractWe present a framework for learning hierarchical policies from demonstrations, using sparse natural language annotations to guide the discovery of reusable skills for autonomous decision-making. We formulate a generative model …...

AbstractContrastive learning has been successfully used for retrieval of semantically aligned sentences, but it often requires large batch sizes or careful engineering to work well. In this paper, we instead …...

AbstractCross-domain aspect-based sentiment analysis (ABSA) aims to perform various fine-grained sentiment analysis tasks on a target domain by transferring knowledge from a source domain. Since labeled data only exists in …...

AbstractWe evaluate the effectiveness of using data augmentation to improve the generalizability of a Named Entity Recognition model for the task of medication identification in clinical notes. We compare disparate …...

AbstractApplying language models to natural language processing tasks typically relies on the representations in the final model layer, as intermediate hidden layer representations are presumed to be less informative. In …...

AbstractWe present our submission to SemEval-2023 Task 10: Explainable Detection of Online Sexism (EDOS). We address all three tasks: Task A consists of identifying whether a post is sexist. If …...