Inducing a Semantically Annotated Lexicon via EM-Based Clustering

Inducing a Semantically Annotated Lexicon via EM-Based Clustering Mats Rooth Stefan Riezler Detlef Prescher Glenn Carroll Franz Beil Institut ffir Maschinelle Sprachverarbeitung University of Stuttgart, Germany Abstract We present a technique for automatic induction of slot annotations for subcategorization frames, based on induction of hidden classes in the EM framework of statistical estimation. The models are empirically evalutated by a general decision test. Induction of slot labeling for subcategoriza- tion frames is accomplished by a further applica- tion of EM, and applied experimentally on frame observations derived from parsing large corpora. We outline an interpretation of the learned rep- resentations as theoretical-linguistic decomposi- tional lexical entries. 1 Introduction An important challenge in computational lin- guistics concerns the construction of large-scale computational lexicons for the numerous natu- ral languages where very large samples of lan- guage use are now available. Resnik (1993) ini- tiated research into the automatic acquisition of semantic selectional restrictions. Ribas (1994) presented an approach which takes into account the syntactic position of the elements whose se- mantic relation is to be acquired. However, those and most of the following approaches require as a prerequisite a fixed taxonomy of semantic rela- tions. This is a problem because (i) entailment hierarchies are presently available for few lan- guages, and (ii) we regard it as an open ques- tion whether and to what degree existing designs for lexical hierarchies are appropriate for repre- senting lexical meaning. Both of these consid- erations suggest the relevance of inductive and experimental approaches to the construction of lexicons with semantic information. This paper presents a method for automatic induction of semantically annotated subcatego- rization frames from unannotated corpora. We use a statistical subcat-induction system which estimates probability distributions and corpus frequencies for pairs of a head and a subcat frame (Carroll and Rooth, 1998). The statistical parser can also collect frequencies for the nomi- nal fillers of slots in a subcat frame. The induc- tion of labels for slots in a frame is based upon estimation of a probability distribution over tu- ples consisting of a class label, a selecting head, a grammatical relation, and a filler head. The class label is treated as hidden data in the EM- framework for statistical estimation. 2 EM-Based Clustering In our clustering approach, classes are derived directly from distributional data--a sample of pairs of verbs and nouns, gathered by pars- ing an unannotated corpus and extracting the fillers of grammatical relations. Semantic classes corresponding to such pairs are viewed as hid- den variables or unobserved data in the context of maximum likelihood estimation from incom- plete data via the EM algorithm. This approach allows us to work in a mathematically well- defined framework of statistical inference, i.e., standard monotonicity and convergence results for the EM algorithm extend to our method. The two main tasks of EM-based clustering are i) the induction of a smooth probability model on the data, and ii) the automatic discovery of class-structure in the data. Both of these aspects are respected in our …

Content truncated to 500 words