A Bayesian Model for Joint Learning of Categories and their Features

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1576–1586, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics A Bayesian Model for Joint Learning of Categories and their Features Lea Frermann and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB l.frermann@ed.ac.uk, mlap@inf.ed.ac.uk Abstract Categories such as ANIMAL or FURNITURE are acquired at an early age and play an impor- tant role in processing, organizing, and con- veying world knowledge. Theories of cat- egorization largely agree that categories are characterized by features such as function or appearance and that feature and category acquisition go hand-in-hand, however previ- ous work has considered these problems in isolation. We present the first model that jointly learns categories and their features. The set of features is shared across categories, and strength of association is inferred in a Bayesian framework. We approximate the learning environment with natural language text which allows us to evaluate performance on a large scale. Compared to highly engi- neered pattern-based approaches, our model is cognitively motivated, knowledge-lean, and learns categories and features which are per- ceived by humans as more meaningful. 1 Introduction Categorization is one of the most basic cognitive functions. It allows individuals to organize their subjective experience of their environment by struc- turing its contents. This ability to group different objects into the same category based on their com- mon characteristics underlies major cognitive activ- ities such as perception, learning, and the use of lan- guage. Global categories (such as FURNITURE or ANIMAL) are shared among members of societies, and influence how we perceive, interact with, and argue about the world. Given its fundamental importance, categoriza- tion is one of the most studied problems in cog- nitive science. The literature is rife with theoret- ical and experimental accounts, as well as model- ing simulations focusing on the emergence, repre- sentation, and learning of categories. Most theo- ries assume that basic level concepts such as dog or chair are characterized by features such as barks or used-for-sitting, and are grouped into cate- gories based on those features. Although the pre- cise grouping mechanism has been subject to con- siderable debate (including arguments in favor of ex- emplars (Nosofsky, 1988), prototypes (Reed, 1972), and category utility (Corter and Gluck, 1992)), it is fairly uncontroversial that categories are associated with featural representations. Experimental studies show that the development of categories and feature learning mutually influ- ence each other (Goldstone et al., 2001; Schyns and Rodet, 1997): concepts are categorized based on their features, but the perception of features is influenced by already established categories, and, like categories, features evolve over time. There is also evidence that features such as barks or runs are grouped into types like behavior (Ahn, 1998; McRae et al., 2005; Spalding and Ross, 2000), and the distribution of feature types varies across cat- egories. For instance, living-things such as ANI- MALS have characteristic …

Content truncated to 500 words