Articles

This paper describes an experimental com- parison of three unsupervised learning al- gorithms that distinguish the sense of an ambiguous word in untagged text. The methods described in this paper, McQuitty's similarity analysis, Ward's minimum-variance method, and the EM algorithm, assign each instance of an am- biguous word to a …...

In this paper, we propose a framework, structured semantic space, as a foundation for word sense disarnbiguation tasks, and present a strategy to identify the correct sense of a word in some context based on the space. The semantic space is a set of multidimensional real-valued vectors, which formally describe …...

This paper presents the results of an em- pirical investigation of temporal reference resolution in scheduling dialogs. The algo- rithm adopted is primarily a linear-recency based approach that does not include a model of global focus. A fully automatic system has been developed and evaluated on unseen test data with …...

Certain applications require that the out- put of an information extraction system be probabilistic, so that a downstream sys- tem can reliably .fuse the output with pos- sibly contradictory information from other sources. In this paper we consider the problem of assigning a probability distri- bution to alternative sets of …...

There has recently been considerable inter- est in the use of lexically-based statistical techniques to resolve prepositional phrase attachments. To our knowledge, however, these investigations have only considered the problem of attaching the first PP, i.e., in a IV NP PP] configuration. In this paper, we consider one technique which …...

Fourteen linguistically-motivated numeri- cal indicators are evaluated for their abil- ity to categorize verbs as either states or events. The values for each indicator are computed automatically across a corpus of text. To improve classification perfor- mance, machine learning techniques are employed to combine multiple indicators. Three machine learning methods …...

Semantic knowledge can be a great asset to natural language processing systems, but it is usually hand-coded for each applica- tion. Although some semantic information is available in general-purpose knowledge bases such as WordNet and Cyc, many ap- plications require domain-specific lexicons that represent words and categories for a particular …...

This paper describes RoboTag, an ad- vanced prototype for a machine learning- based multilingual information extraction system. First, we describe a general client/server architecture used in learning from observation. Then we give a detailed description of our novel decision-tree tag- ging approach. RoboTag performance for the proper noun tagging task …...

Automatic segmentation of text into min- imal content-bearing units is an unsolved problem even for languages like English. Spaces between words offer an easy first ap- proximation, but this approximation is not good enough for machine translation (MT), where many word sequences are not trans- lated word-for-word. This paper presents …...

We investigate the effects of lexicon size and stopwords on Chinese information retrieval using our method of short-word segmentation based on simple language usage rules and statistics. These rules allow us to employ a small lexicon of only 2,175 entries and provide quite admirable retrieval results. It is noticed that …...

This paper presents a corpus-based method to assign grammatical subject/object re- lations to ambiguous German constructs. It makes use of an unsupervised learning procedure to collect training and test data, and the back-off model to make assignment decisions. 1...

We consider the use of language models whose size and accuracy are intermedi- ate between different order n-gram models. Two types of models are studied in partic- ular. Aggregate Markov models are class- based bigram models in which the map- ping from words to classes is probabilis- tic. Mixed-order Markov …...

Few attention has been paid to terminol- ogy extraction for what concerns the pos- sibilities it offers to corpus linguistics and lexical acquisition. The problem of detect- ing terms in textual corpora has been ap- proached in a complex framework. Termi- nology is seen as the acquisition of domain specific …...

This paper addresses issues in automated treebank construction. We show how stan- dard part-of-speech tagging techniques ex- tend to the more general problem of struc- tural annotation, especially for determi- ning grammatical functions and syntactic categories. Annotation is viewed as an in- teractive process where manual and auto- matic processing …...

The increasing availability of corpora anno- tated for linguistic structure prompts the question: if we have the same texts, anno- tated for phrase structure under two dif- ferent schemes, to what extent do the an- notations agree on structuring within the text? We suggest the term tree alignment to indicate …...

Learning problems in the text processing domain often map the text to a space whose dimensions are the measured fea- tures of the text, e.g., its words. Three characteristic properties of this domain are (a) very high dimensionality, (b) both the learned concepts and the instances reside very sparsely in …...

This paper introduces a new statistical ap- proach to partitioning text automatically into coherent segments. Our approach en- lists both short-range and long-range lan- guage models to help it sniff out likely sites of topic changes in text. To aid its search, the system consults a set of simple lexical …...

Although Minimum Distance Parsing (MDP) offers a theoretically attractive so- lution to the problem of extragrammat- icality, it is often computationally infea- sible in large scale practical applications. In this paper we present an alternative ap- proach where the labor is distributed be- tween a more restrictive partial parser and …...

We present a variation on classic beam thresholding techniques that is up to an or- der of magnitude faster than the traditional method, at the same performance level. We also present a new thresholding technique, global thresholding, which, combined with the new beam thresholding, gives an ad- ditional factor of …...

This paper presents a statistical parser for natural language that obtains a parsing accuracy--roughly 87% precision and 86% recall--which surpasses the best previously published results on the Wall St. Journal domain. The parser itself requires very lit- tle human intervention, since the informa- tion it uses to make parsing decisions …...