Distributional Modeling on a Diet: One-shot Word Learning from Text Only

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 204–213, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Distributional Modeling on a Diet: One-shot Word Learning from Text Only Su Wang♠ Stephen Roller♣ Katrin Erk♠ ♠Department of Linguistics, ♣Department of Computer Science The University of Texas at Austin shrekwang@utexas.edu roller@cs.utexas.edu, katrin.erk@mail.utexas.edu Abstract We test whether distributional models can do one-shot learning of definitional prop- erties from text only. Using Bayesian models, we find that first learning overar- ching structure in the known data, regular- ities in textual contexts and in properties, helps one-shot learning, and that individ- ual context items can be highly informa- tive. Our experiments show that our model can learn properties from a single expo- sure when given an informative utterance. 1 Introduction When humans encounter an unknown word in text, even with a single instance, they can often infer approximately what it means, as in this example from Lazaridou et al. (2014): We found a cute, hairy wampimuk sleep- ing behind the tree. People who hear this sentence typically guess that a wampimuk is an animal, or even that it is a mammal. Distributional models, which describe the meaning of a word in terms of its observed contexts (Turney and Pantel, 2010), have been suggested as a model for how humans learn word meanings (Landauer and Dumais, 1997). How- ever, distributional models typically need hun- dreds of instances of a word to derive a high- quality representation for it, while humans can of- ten infer a passable meaning approximation from one sentence only (as in the above example). This phenomenon is known as fast mapping (Carey and Bartlett, 1978), Our primary modeling objective in this paper is to explore a plausible model for fast- mapping learning from textual context. While there is preliminary evidence that fast mapping can be modeled distributionally (Lazari- dou et al., 2016), it is unclear what enables it. How do humans infer word meanings from so little data? This question has been studied for grounded word learning, when the learner per- ceives an object in non-linguistic context that cor- responds to the unknown word. The literature emphasizes the importance of learning general knowledge or overarching structure, which we de- fine as the information that is learned by accumu- lation across concepts (e.g. regularities in property co-occurrence), across all concepts (Kemp et al., 2007), In grounded word learning, overarching structure that has been proposed includes knowl- edge about which properties. For example knowl- edge about which properties are most important to object naming (Smith et al., 2002; Colunga and Smith, 2005), or a taxonomy of concepts (Xu and Tenenbaum, 2007). In this paper we study models for fast mapping in word learning1 from textual context alone, us- ing probabilistic distributional models. Our task differs from the grounded case in that we do not perceive any object labeled by the unknown word. In that context, learning word meaning means learning the associated definitional properties and their weights (see …

Content truncated to 500 words