Minimally Supervised Classification to Semantic Categories using Automatically Acquired Symmetric Patterns

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1612–1623, Dublin, Ireland, August 23-29 2014. Minimally Supervised Classification to Semantic Categories using Automatically Acquired Symmetric Patterns Roy Schwartz1 Roi Reichart2 1Institute of Computer Science, The Hebrew University {roys02|arir}@cs.huji.ac.il 2Technion IIT roiri@ie.technion.ac.il Ari Rappoport1 Abstract Classifying nouns into semantic categories (e.g., animals, food) is an important line of research in both cognitive science and natural language processing. We present a minimally supervised model for noun classification, which uses symmetric patterns (e.g., “X and Y”) and an iterative variant of the k-Nearest Neighbors algorithm. Unlike most previous works, we do not use a predefined set of symmetric patterns, but extract them automatically from plain text, in an unsu- pervised manner. We experiment with four semantic categories and show that symmetric patterns constitute much better classification features compared to leading word embedding methods. We further demonstrate that our simple k-Nearest Neighbors algorithm outperforms two state-of- the-art label propagation alternatives for this task. In experiments, our model obtains 82%-94% accuracy using as few as four labeled examples per category, emphasizing the effectiveness of simple search and representation techniques for this task. 1 Introduction The role of language is to express meaning. In the field of NLP, there has been an increasingly grow- ing number of approaches that deal with semantics. Among these are vector space models (Turney and Pantel, 2010; Baroni and Lenci, 2010), lexical acquisition (Hearst, 1992; Dorow et al., 2005; Davidov and Rappoport, 2006), universal cognitive conceptual annotation (Abend and Rappoport, 2013) and au- tomatic induction of feature representations (Collobert et al., 2011). In this paper, we utilize extremely weak supervision to classify words into fundamental cognitive semantic categories. There are several types of semantic categories expressed by languages, e.g., objects, actions, and properties. We follow human development, acquiring coarse-grained categories and distinctions before detailed ones (Mandler, 2004). Specifically, we focus on the major class of concrete “things” (Langacker, 2008, Ch. 4), roughly corresponding to nouns – the main participants in linguistic clauses – that are universally present in the semantics of virtually all languages (Dixon, 2005). Most works on noun classification to semantic categories require large amounts of human annotation to build training corpora for supervised algorithms (Bowman and Chopra, 2012; Moore et al., 2013) or rely on language-specific resources such as WordNet (Evans and Orˇasan, 2000; Orˇasan and Evans, 2007). Such heavy supervision is labor intensive and makes these models domain and language dependent. Our reasoning is that weak supervision is highly valuable for semantic categorization, as it can com- pensate for the lack of input from the senses in text corpora. Our model therefore performs semantic category classification using only a small number of labeled seed words per category. The experiments we conduct show that such weak supervision is sufficient to construct a high quality classifier. A key component of our model is the application of symmetric patterns. We define patterns to be sequences of words and wildcards (e.g., “X is a dog”, “both …

Content truncated to 500 words