USAAR-WLV: Hypernym Generation with Deep Neural Nets

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 932–937, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics USAAR-WLV: Hypernym Generation with Deep Neural Nets Liling Tan, Rohit Guptaα and Josef van Genabithβ Universit¨at des Saarlandes / Campus A2.2, Saarbr¨ucken, Germany University of Wolverhamptonα / Wulfruna Street, Wolverhampton, UK Deutsches Forschungszentrum f¨ur Knstliche Intelligenzβ / Stuhlsatzenhausweg, Saarbr¨ucken, Germany alvations@gmail.com, r.gupta@wlv.ac.uk, josef.van genabith@dfki.de Abstract This paper describes the USAAR-WLV tax- onomy induction system that participated in the Taxonomy Extraction Evaluation task of SemEval-2015. We extend prior work on using vector space word embedding models for hypernym-hyponym extraction by simpli- fying the means to extract a projection matrix that transforms any hyponym to its hypernym. This is done by making use of function words, which are usually overlooked in vector space approaches to NLP. Our system performs best in the chemical domain and has achieved com- petitive results in the overall evaluations. 1 Introduction Traditionally, broad-coverage semantic taxonomies such as CYC (Lenat, 1995) and WordNet ontol- ogy (Miller, 1995) have been manually created with much effort and yet they suffer from coverage spar- sity. This motivated the move towards unsupervised approaches to extract structured relational knowl- edge from texts (Lin and Pantel, 2001; Snow et al., 2006; Velardi et al., 2013).1 Previous work in taxonomy extraction focused on rule-based, clustering and graph-based approaches. Although vector space approaches are popular in current NLP researches, ontology induction studies have yet to catch on the frenzy. Fu et al. (2014) proposed a vector space approach to hypernym- hyponym identification using word embeddings that 1For the rest of the paper, taxonomy and ontology will be used interchangeably to refer to a hierarchically structure that organizes a list of concepts. trains a projection matrix2 that converts a hyponym vector to its hypernym. However, their approach requires an existing hypernym-hyponym pairs for training before discovering new pairs. Our system submitted to the SemEval-2015 tax- onomy building task is most similar to the approach by Fu et al. (2014) in using word embeddings pro- jections to identify hypernym-hyponym pairs. As opposed to previous method our method does not re- quires prior taxonomical knowledge. Instead of training a projection matrix, we capi- talize on the fact that hypernym-hyponym pair often occurs in a sentence with an ‘is a’ phrase, e.g. “The goldfish (Carassius auratus auratus) is a freshwa- ter fish”.3 Intuitively, if we single-tokenize the ‘is a’ phrase prior to training a vector space, we can make use of the vector that represents the phrase in capturing a hypernym-hyponym pair as such the multiplication of v(goldfish) and v(is-a) will be similar to the cross product v(fish) (v(goldfish)×v(is-a) ≈ v(fish)). There is little or no previous work that manipu- lates non-content word vectors in vector space mod- els studies for natural language processing. Of- ten, non-content words4 were implicitly incorpo- rated into the vector space models by means of syn- tactic frames (Sarmento et al., 2009) or syntactic parses (Thater et al., 2010). Our main contribution …

Content truncated to 500 words