Maximum Likelihood Estimation

Speaker-independent system is desirable in many applications where speaker-specific data do not exist. However, if speaker- dependent data are available, the system could be adapted to the specific speaker such …...

We discuss maximum a posteriori estimation of continuous density hid- den Markov models (CDHMM). The classical MLE reestimation algorithms, namely the forward-backward algorithm and the segmental k-means algo- rithm, are …...

We compare four similarity-based esti- mation methods against back-off and maximum-likelihood estimation meth- ods on a pseudo-word sense disam- biguation task in which we controlled for both unigram and bigram …...

This paper proposes a new discriminative training method, called minimum sample risk (MSR), of estimating parameters of language models for text input. While most existing discriminative training methods use a …...

Lasso is a regularization method for pa- rameter estimation in linear models. It op- timizes the model parameters with respect to a loss function subject to model com- plexities. This …...

Unsupervised learning of linguistic structure is a difficult problem. A common approach is to define a generative model and max- imize the probability of the hidden struc- ture given the …...

We address the problem of training the free parameters of a statistical machine transla- tion system. We show significant improve- ments over a state-of-the-art minimum er- ror rate training baseline …...

This paper presents two Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of probabilistic con- text free grammars (PCFGs) from ter- minal strings, providing an alternative to maximum-likelihood estimation …...

Paraphrase patterns are useful in paraphrase recognition and generation. In this paper, we present a pivot approach for extracting para- phrase patterns from bilingual parallel cor- pora, whereby the English …...

We introduce a latent activity model for work- place emails, positing that communication at work is purposeful and organized by activities. We pose the problem as probabilistic infer- ence in …...

The neural network joint model of transla- tion or NNJM (Devlin et al., 2014) combines source and target context to produce a power- ful translation feature. However, its softmax layer …...

We propose minimum risk training for end-to-end neural machine translation. Unlike conventional maximum likelihood estimation, minimum risk training is ca- pable of optimizing model parameters di- rectly with respect to …...

AbstractLanguage models are generally trained on data spanning a wide range of topics (e.g., news, reviews, fiction), but they might be applied to an a priori unknown target distribution (e.g., …...

AbstractNeural language models are usually trained using Maximum-Likelihood Estimation (MLE). The corresponding objective function for MLE is derived from the Kullback-Leibler (KL) divergence between the empirical probability distribution representing the …...

AbstractLanguage models trained with Maximum Likelihood Estimation (MLE) have been considered as a mainstream solution in Natural Language Generation (NLG) for years. Recently, various approaches with Generative Adversarial Nets (GANs) …...

AbstractIn this paper, we study sequence-to-sequence (S2S) keyphrase generation models from the perspective of diversity. Recent advances in neural natural language generation have made possible remarkable progress on the task …...

AbstractNeural dialogue response generation has gained much popularity in recent years. Maximum Likelihood Estimation (MLE) objective is widely adopted in existing dialogue model learning. However, models trained with MLE objective …...

AbstractRecent studies have revealed a number of pathologies of neural machine translation (NMT) systems. Hypotheses explaining these mostly suggest there is something fundamentally wrong with NMT as a model or …...

AbstractThe uniform information density (UID) hypothesis, which posits that speakers behaving optimally tend to distribute information uniformly across a linguistic signal, has gained traction in psycholinguistics as an explanation for …...

AbstractThe discrepancy between maximum likelihood estimation (MLE) and task measures such as BLEU score has been studied before for autoregressive neural machine translation (NMT) and resulted in alternative training algorithms …...

AbstractConversational Question Simplification (CQS) aims to simplify self-contained questions into conversational ones by incorporating some conversational characteristics, e.g., anaphora and ellipsis. Existing maximum likelihood estimation based methods often get trapped …...

AbstractWhen training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers …...

AbstractDespite recent progress in abstractive summarization, systems still suffer from faithfulness errors. While prior work has proposed models that improve faithfulness, it is unclear whether the improvement comes from an …...

AbstractAutomatic medical question summarization can significantly help the system to understand consumer health questions and retrieve correct answers. The Seq2Seq model based on maximum likelihood estimation (MLE) has been applied …...

AbstractGenerating temporally-ordered event sequences in texts is important to natural language processing. Two emerging tasks in this direction are temporal event ordering (rearranging the set of events to correct order) …...

AbstractKeyphrase generation aims to generate a set of condensed phrases given a source document. Although maximum likelihood estimation (MLE) based keyphrase generation methods have shown impressive performance, they suffer from …...

AbstractPersuasive conversations for a social cause often require influencing other person’s attitude or intention that may fail even with compelling arguments. The use of emotions and different types of polite …...

AbstractSince the meaning representations are detailed and accurate annotations which express fine-grained sequence-level semtantics, it is usually hard to train discriminative semantic parsers via Maximum Likelihood Estimation (MLE) in an …...

AbstractSupervised learning in Neural Machine Translation (NMT) standardly follows a teacher forcing paradigm where the conditioning context in the model’s prediction is constituted by reference tokens, instead of its own …...

AbstractSummarization models often generate text that is poorly calibrated to quality metrics because they are trained to maximize the likelihood of a single reference (MLE). To address this, recent work …...