Articles

We investigate the performance of the Structured Language Model (SLM) in terms of perplexity (PPL) when its compo- nents are modeled by connectionist mod- els. The connectionist models use a dis- tributed representation of the items in the history and make much better use of contexts than currently used interpolated …...

This paper describes a fast algorithm that se- lects features for conditional maximum en- tropy modeling. Berger et al. (1996) presents an incremental feature selection (IFS) algo- rithm, which computes the approximate gains for all candidate features at each selection stage, and is very time-consuming for any problems with large …...

Discriminative models have been of inter- est in the NLP community in recent years. Previous research has shown that they are advantageous over generative mod- els. In this paper, we investigate how dif- ferent objective functions and optimiza- tion methods affect the performance of the classifiers in the discriminative learning …...

A maximum entropy (ME) model is usu- ally estimated so that it conforms to equal- ity constraints on feature expectations. However, the equality constraint is inap- propriate for sparse and therefore unre- liable features. This study explores an ME model with box-type inequality con- straints, where the equality can be …...

Opinion question answering is a challenging task for natural language processing. In this paper, we discussa necessarycomponentfor an opinion ques- tion answering system: separating opinions from fact, at both the document and sentence level. We present a Bayesian classifier for discriminating be- tween documents with a preponderanceof opinions such as …...

A Natural Language Generation system produces text using as input semantic data. One of its very first tasks is to decide which pieces of information to convey in the output. This task, called Content Se- lection, is quite domain dependent, requir- ing considerable re-engineering to trans- port the system from …...

Successful application of multi-view co- training algorithms relies on the ability to factor the available features into views that are compatible and uncorrelated. This can potentially preclude their use on problems such as coreference resolution that lack an obvious feature split. To bootstrap coref- erence classifiers, we propose and eval- …...

This paper describes log-linear pars- ing models for Combinatory Categorial Grammar (CCG). Log-linear models can easily encode the long-range dependen- cies inherent in coordination and extrac- tion phenomena, which CCG was designed to handle. Log-linear models have pre- viously been applied to statistical pars- ing, under the assumption that all …...

We propose the use of Lexicalized Tree Adjoining Grammar (LTAG) as a source of features that are useful for reranking the output of a statistical parser. In this paper, we extend the notion of a tree ker- nel over arbitrary sub-trees of the parse to the derivation trees and derived …...

This paper compares a range of methods for classifying words based on linguis- tic diagnostics, focusing on the task of learning countabilities for English nouns. We propose two basic approaches to feature representation: distribution-based representation, which simply looks at the distribution of features in the cor- pus data, and agreement-based …...

In this paper we explore the variation of sentences as a function of the sentence number. We demonstrate that while the entropy of the sentence increases with the sentence number, it decreases at the para- graph boundaries in accordance with the Entropy Rate Constancy principle (intro- duced in related work). …...

This paper presents a bootstrapping process that learns linguistically rich extraction pat- terns for subjective (opinionated) expressions. High-precision classifiers label unannotated data to automatically create a large training set, which is then given to an extraction pattern learning algorithm. The learned patterns are then used to identify more subjective sentences. …...

The development of FrameNet, a large database of semantically annotated sen- tences, has primed research into statistical methods for semantic tagging. We ad- vance previous work by adopting a Maximum Entropy approach and by using previous tag information to find the high- est probability tag sequence for a given sentence. …...

We use deep linguistic features to predict semantic roles on syntactic arguments, and show that these perform considerably better than surface-oriented features. We also show that predicting labels from a “lightweight” parser that generates deep syntactic features performs comparably to using a full parser that generates only sur- face syntactic …...

We address the problem of sentence align- ment for monolingual corpora, a phe- nomenon distinct from alignment in par- allel corpora. Aligning large compara- ble corpora automatically would provide a valuable resource for learning of text-to- text rewriting rules. We incorporate con- text into the search for an optimal align- …...

This paper explores the problem of find- ing non-local dependencies. First, we isolate a set of features useful for this task. Second, we develop both a two-step approach which combines a trace tagger with a state-of-the-art lexicalized parser and a one-step approach which finds non- local dependencies while parsing. We …...

We present a system for automatically identifying PropBank-style semantic roles based on the output of a statistical parser for Combinatory Categorial Grammar. This system performs at least as well as a system based on a traditional Treebank parser, and outperforms it on core argu- ment roles. 1...

We define, implement and evaluate a novel model for statistical machine translation, which is based on shal- low syntactic analysis (part-of-speechtagging and phrase chunking) in both the source and target languages. It is able to model long-distance constituent motion and other syntactic phenomena without requiring a full parse in either …...

In this paper, we describe a phrase-based unigram model for statistical machine translation that uses a much simpler set of model parameters than similar phrase- based models. The units of translation are blocks – pairs of phrases. During decod- ing, we use a block unigram model and a word-based trigram …...

Language variationists study how lan- guages vary along geographical or so- cial lines or along lines of age and gender. Variationist data is available and challenging, in particular for DI- ALECTOLOGY, the study of geograph- ical variation, which will be the focus of this paper, although we present ap- proaches …...