Incremental Bayesian Learning of Semantic Categories

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 249–258, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics Incremental Bayesian Learning of Semantic Categories Lea Frermann and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB l.frermann@ed.ac.uk, mlap@inf.ed.ac.uk Abstract Models of category learning have been ex- tensively studied in cognitive science and primarily tested on perceptual abstractions or artificial stimuli. In this paper we focus on categories acquired from natural lan- guage stimuli, that is words (e.g., chair is a member of the FURNITURE category). We present a Bayesian model which, un- like previous work, learns both categories and their features in a single process. Our model employs particle filters, a sequential Monte Carlo method commonly used for approximate probabilistic inference in an incremental setting. Comparison against a state-of-the-art graph-based approach re- veals that our model learns qualitatively better categories and demonstrates cogni- tive plausibility during learning. 1 Introduction Considerable psychological research has shown that people reason about novel objects they en- counter by identifying the category to which these objects belong and extrapolating from their past experiences with other members of that category (Smith and Medin, 1981). Categorization is a clas- sic problem in cognitive science, underlying a va- riety of common mental tasks including percep- tion, learning, and the use of language. Given its fundamental nature, categorization has been extensively studied both experimentally and in simulations. Indeed, numerous models ex- ist as to how humans categorize objects ranging from strict prototypes (categories are represented by a single idealized member which embodies their core properties; e.g., Reed 1972) to full ex- emplar models (categories are represented by a list of previously encountered members; e.g., Nosof- sky 1988) and combinations of the two (e.g., Grif- fiths et al. 2007). A common feature across dif- ferent studies is the use of stimuli involving real- world objects (e.g., children’s toys; Starkey 1981), perceptual abstractions (e.g., photographs of ani- mals; Quinn and Eimas 1996), or artificial ones (e.g., binary strings, dot patterns or geometric shapes; Medin and Schaffer 1978; Posner and Keele 1968; Bomba and Siqueland 1983). Most existing models focus on adult categorization, in which it is assumed that a large number of cate- gories have already been learnt (but see Anderson 1991 and Griffiths et al. 2007 for exceptions). In this work we focus on categories acquired from natural language stimuli (i.e., words) and investigate how the statistics of the linguistic en- vironment (as approximated by large corpora) in- fluence category formation (e.g., chair and ta- ble are FURNITURE whereas peach and apple are FRUIT1). The idea of modeling categories using words as a stand-in for their referents has been previously used to explore categorization-related phenomena such as semantic priming (Cree et al., 1999) and typicality rating (Voorspoels et al., 2008), to evaluate prototype and exemplar mod- els (Storms et al., 2000), and to simulate early lan- guage category acquisition (Fountain …

Content truncated to 500 words