A Statistical Approach towards Unknown Word Type Prediction for Deep Grammars

Proceedings of the Australasian Language Technology Workshop 2005, pages 24–31, Sydney, Australia, December 2005. A Statistical Approach towards Unknown Word Type Prediction for Deep Grammars Yi Zhang and Valia Kordoni Department of Computational Linguistics Saarland University Saarbr¨ucken, Germany, D-66041 Abstract This paper presents a statistical approach to un- known word type prediction for a deep HPSG grammar. Our motivation is to enhance robust- ness in deep processing. With a predictor which predicts lexical types for unknown words ac- cording to the context, new lexical entries can be generated on the fly. The predictor is a maximum entropy based classifier trained on a HPSG treebank. By exploring various feature templates and the feedback from parse disam- biguation results, the predictor achieves preci- sion over 60%. The models are general enough to be applied to other constraint-based gram- mar formalisms. 1 Introduction Deep processing delivers fine-grained syntactic and semantic analyses which are desirable for advanced NLP applications. However, speci- ficity and robustness are the major difficulties that deep processing has encountered for years. Unlike shallow methods, which in most cases deliver an expected number of analyses, the number of output from deep processing is usu- ally unpredictable, especially for open texts. The specificity problem arises when there are more analyses generated than expected. The analyses might be linguistically sound, but prac- tically uninteresting for real applications. Re- cently, with more deep processing resources made available (Oepen et al., 2002), the speci- ficity problem is being alleviated with statistical parse selection models (Toutanova et al., 2002). As to robustness, more open questions remain to be investigated. A deep grammar is normally a complicated rule system. Whenever the input varies, even slightly, beyond the grammar de- velopers’ expectations, the output becomes un- predictable. Closer studies of deep grammars have shown that lexicon coverage is one of the major barri- ers preventing deep grammars from being used for open text processing. Take the LinGO English Resource Grammar (ERG) (Copestake and Flickinger, 2000), for instance. The gram- mar has been developed for more than 10 years, and currently contains about 22K lexicon en- tries. A recent test on the BNC corpus reported that only 32% of the strings have full lexical span, of which 57% get at least one parse (Bald- win et al., 2004). About 40% of the parsing failures are due to lexicon missing. Lexicalized deep grammars rely on knowledge-rich lexicon. However, the construction of a lexicon with de- cent coverage requires a huge amount of human effort and considerable linguistic proficiency. A widely adopted approach towards robust deep processing is to integrate shallow meth- ods (Callmeier et al., 2004). However, most re- cent approaches still work on various fall-back strategies. When a deep processing component fails to deliver output, intermediate or shallow components are invoked to provide compatible analyses. Practically valid, this approach does not directly help to enhance the robustness of deep processing itself. Inspired by the statistical approaches in parse selection (Toutanova et al., 2002), we pro- pose a statistical approach …

Content truncated to 500 words