Unsupervised Information Extraction with Distributional Prior Knowledge

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 814–824, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Unsupervised Information Extraction with Distributional Prior Knowledge Cane Wing-ki Leung1, Jing Jiang1, Kian Ming A. Chai2, Hai Leong Chieu2, Loo-Nin Teow2 1School of Information Systems, Singapore Management University, Singapore 2DSO National Laboratories, Singapore {caneleung,jingjiang}@smu.edu.sg, {ckianmin,chaileon,tloonin}@dso.org.sg Abstract We address the task of automatic discovery of information extraction template from a given text collection. Our approach clusters candi- date slot fillers to identify meaningful tem- plate slots. We propose a generative model that incorporates distributional prior knowl- edge to help distribute candidates in a docu- ment into appropriate slots. Empirical results suggest that the proposed prior can bring sub- stantial improvements to our task as compared to a K-means baseline and a Gaussian mixture model baseline. Specifically, the proposed prior has shown to be effective when coupled with discriminative features of the candidates. 1 Introduction Information extraction (IE) is the task of extract- ing information from natural language texts to fill a database record following a structure called a tem- plate. Such templates are usually defined based on the domain of interest. For example, the do- main in the Sixth Message Understanding Confer- ence (MUC-6, 1995) is management succession, and the pre-defined template consists of the slots posi- tion, the person leaving, the person joining, and the organization. Previous research on IE often requires the pre- definition of templates. Template construction is usually done manually by domain experts, and an- notated documents are often created to facilitate su- pervised learning approaches to IE. However, both manual template construction and data annotation are labor-intensive. More importantly, templates and annotated data usually cannot be re-used in new do- mains due to domain dependency. It is therefore nat- ural to consider the problem of unsupervised tem- plate induction and information extraction. This is the topic of this paper. There have been a few previous attempts to ad- dress the unsupervised IE problem (Shinyama and Sekine, 2006; Sekine, 2006; Rosenfeld and Feld- man, 2006; Filatova et al., 2006). These approaches have a commonality: they try to cluster candidate slot fillers, which are often nouns and noun phrases, into slots of the template to be constructed. How- ever, most of them have neglected the following im- portant observation: a single document or text seg- ment tends to cover different slots rather than re- dundantly fill the same slot. In other words, during clustering, candidates within the same text segment should be more likely to be distributed into different clusters. In this paper, we propose a generative model that incorporates this distributional prior knowledge. We define a prior distribution over the possible label assignments in a document or a text segment such that a more diversified label assignment is preferred. This prior is based on the Poisson distribution. We also compare a number of generative models for generating slot fillers and find that the Gaussian mix- ture model is the best. We then …

Content truncated to 500 words