Оценка контраста шума

We explore the application of neural language models to machine translation. We develop a new model that combines the neural proba- bilistic language model of Bengio et al., rec- tified …...

This study proposes a word alignment model based on a recurrent neural net- work (RNN), in which an unlimited alignment history is represented by re- currently connected hidden layers. We …...

This paper presented the joined submis- sion of KIT and LIMSI to the English to German translation task of WMT 2015. In this year submission, we integrated a neu- ral …...

The neural network joint model of transla- tion or NNJM (Devlin et al., 2014) combines source and target context to produce a power- ful translation feature. However, its softmax layer …...

. This paper proposes a new discriminative framework to train continuous-space trans- lation models based. This framework relies on the definition of a new objective function that al- lows us …...

We present a simple algorithm to efficiently train language models with noise-contrastive estimation (NCE) on graphics processing units (GPUs). Our NCE-trained language models achieve significantly lower perplexity on the One …...

Training neural network language mod- els over large vocabularies is computa- tionally costly compared to count-based models such as Kneser-Ney. We present a systematic comparison of neural strate- gies to …...

AbstractIn this study, we introduce a new approach for learning language models by training them to estimate word-context pointwise mutual information (PMI), and then deriving the desired conditional probabilities from …...

AbstractNoise-Contrastive Estimation (NCE) is a learning criterion that is regularly used to train neural language models in place of Maximum Likelihood Estimation, since it avoids the computational bottleneck caused by …...

AbstractSelf-normalizing discriminative models approximate the normalized probability of a class without having to compute the partition function. In the context of language modeling, this property is particularly appealing as it …...

Continuous representations of linguistic struc- tures are an important part of modern natu- ral language processing systems. Despite the diversity, most of the existing log-multilinear embedding models are organized under …...

AbstractIn this work, we explore joint energy-based model (EBM) training during the finetuning of pretrained text encoders (e.g., Roberta) for natural language understanding (NLU) tasks. Our experiments show that EBM …...

AbstractThe choice of negative examples is important in noise contrastive estimation. Recent works find that hard negatives—highest-scoring incorrect examples under the model—are effective in practice, but they are used without …...

AbstractEvent trigger detection, entity mention recognition, event argument extraction, and relation extraction are the four important tasks in information extraction that have been performed jointly (Joint Information Extraction - JointIE) …...