Latent Variable Models of Concept-Attribute Attachment

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 620–628, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP Latent Variable Models of Concept-Attribute Attachment Joseph Reisinger∗ Department of Computer Sciences The University of Texas at Austin Austin, Texas 78712 joeraii@cs.utexas.edu Marius Pas¸ca Google Inc. 1600 Amphitheatre Parkway Mountain View, California 94043 mars@google.com Abstract This paper presents a set of Bayesian methods for automatically extending the WORDNET ontology with new concepts and annotating existing concepts with generic property fields, or attributes. We base our approach on Latent Dirichlet Al- location and evaluate along two dimen- sions: (1) the precision of the ranked lists of attributes, and (2) the quality of the attribute assignments to WORDNET concepts. In all cases we find that the principled LDA-based approaches outper- form previously proposed heuristic meth- ods, greatly improving the specificity of attributes at each concept. 1 Introduction We present a Bayesian approach for simultane- ously extending Is-A hierarchies such as those found in WORDNET (WN) (Fellbaum, 1998) with additional concepts, and annotating the resulting concept graph with attributes, i.e., generic prop- erty fields shared by instances of that concept. Ex- amples of attributes include “height” and “eye- color” for the concept Person or “gdp” and “pres- ident” for Country. Identifying and extracting such attributes relative to a set of flat (i.e., non- hierarchically organized) labeled classes of in- stances has been extensively studied, using a vari- ety of data, e.g., Web search query logs (Pas¸ca and Van Durme, 2008), Web documents (Yoshinaga and Torisawa, 2007), and Wikipedia (Suchanek et al., 2007; Wu and Weld, 2008). Building on the current state of the art in at- tribute extraction, we propose a model-based ap- proach for mapping flat sets of attributes anno- tated with class labels into an existing ontology. This inference problem is divided into two main components: (1) identifying the appropriate par- ent concept for each labeled class and (2) learning ∗Contributions made during an internship at Google. the correct level of abstraction for each attribute in the extended ontology. For example, consider the task of annotating WN with the labeled class re- naissance painters containing the class instances Pisanello, Hieronymus Bosch, and Jan van Eyck and associated with the attributes “famous works” and “style.” Since there is no WN concept for renaissance painters, the latter would need to be mapped into WN under, e.g., Painter. Further- more, since “famous works” and “style” are not specific to renaissance painters (or even the WN concept Painter), they should be placed at the most appropriate level of abstraction, e.g., Artist. In this paper, we show that both of these goals can be realized jointly using a probabilistic topic model, namely hierarchical Latent Dirichlet Allo- cation (LDA) (Blei et al., 2003b). There are three main advantages to using a topic model as the annotation procedure: (1) Unlike hi- erarchical clustering (Duda et al., 2000), the at- tribute distribution at a concept node is not com- posed of the …

Content truncated to 500 words