Statistical Language Model

We present a novel method for segmenting the input sentence into words and assigning parts of speech to the words. It consists of a statistical language model and an efficient …...

We describe the use of energy function op- timisation in very shallow syntactic pars- ing. The approach can use linguistic rules and corpus-based statistics, so the strengths of both linguistic …...

To understand a speaker's turn of a con- versation, one needs to segment it into in- tonational phrases, clean up any speech re- pairs that might have occurred, and iden- …...

A statistical language model may be used to segment a data sequence by thresholding its instantaneous entropy. In this paper we de- scribe how this process works, and we apply …...

AbstractIn this paper we describe a lexical disambiguation algorithm based on a statistical language model we call maximum likelihood disambiguation. The maximum likelihood method depends solely on the target language. …...

In this paper we present some lessons learned from building viz s la, the keyword search and topic classification system used on the largest Hungarian portal, [ origo .hu]. Based …...

We present experimental evidence that providing naive users of a spoken dia- logue system with immediate help mes- sages related to their out-of-coverage ut- terances improves their success in using …...

Statistical language models using n-gram approach have been under the criticism of neglecting large-span syntactic-semantic in- formation that influences the choice of the next word in a language. One of …...

This paper describes the development of a statistical machine translation system based on the Moses decoder for the 2007 WMT shared tasks. Several different translation strategies were explored. We also …...

Data sparsity is one of the main factors that make word sense disambiguation (WSD) difficult. To overcome this problem we need to find effective ways to use resources other than …...

In this paper we propose a novel statistical language model to capture long-range se- mantic dependencies. Specifically, we ap- ply the concept of semantic composition to the problem of constructing …...

In this paper, we present an efficient query selection algorithm for the retrieval of web text data to augment a statistical language model (LM). The number of retrieved rel- evant …...

In this paper we present a novel approach to entity linking based on a statistical lan- guage model-based information retrieval with query expansion. We use both local con- texts and …...

AbstractIn the implementation of a surface realisation engine, many of the computational techniques seen in other AI fields have been widely applied. Among these, the use of statistical methods has …...

This paper describes an Example-Based Ma- chine Translation prototype and presents an evaluation of the impact of using a domain- specific vocabulary on its performance. This prototype is based on …...

Methods to predict the effort needed to post-edit a given machine translation (MT) output are seen as a promising direction to making MT more useful in the translation industry. Despite …...

AbstractExtracting location names from informal and unstructured social media data requires the identification of referent boundaries and partitioning compound names. Variability, particularly systematic variability in location names (Carroll, 1983), challenges …...