A Neural Network Approach to Selectional Preference Acquisition

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 26–35, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics A Neural Network Approach to Selectional Preference Acquisition Tim Van de Cruys IRIT & CNRS Toulouse, France tim.vandecruys@irit.fr Abstract This paper investigates the use of neural networks for the acquisition of selectional preferences. Inspired by recent advances of neural network models for NLP applica- tions, we propose a neural network model that learns to discriminate between felici- tous and infelicitous arguments for a par- ticular predicate. The model is entirely un- supervised – preferences are learned from unannotated corpus data. We propose two neural network architectures: one that han- dles standard two-way selectional prefer- ences and one that is able to deal with multi-way selectional preferences. The model’s performance is evaluated on a pseudo-disambiguation task, on which it is shown to achieve state of the art perfor- mance. 1 Introduction Predicates often have a semantically motivated pref- erence for particular arguments. Compare for ex- ample the sentences in (1) and (2). (1) The vocalist sings a ballad. (2) The exception sings a tomato. Most language users would have no problems ac- cepting the first sentence as well-formed: a vocalist can be expected to sing, and a ballad is something that can be sung. The same language users, how- ever, would likely consider the second sentence to be ill-formed: an exception is not supposed to sing, nor is a tomato something that is typically sung. Within the field of natural language processing, this inclination of predicates to select for particular arguments is known as selectional preference. The automatic acquisition of selectional prefer- ences has been a popular research subject within the field of natural language processing. An auto- matically acquired selectional preference resource is a versatile tool for numerous NLP applications, such as semantic role labeling (Gildea and Jurafsky, 2002), word sense disambiguation (McCarthy and Carroll, 2003), and metaphor processing (Shutova et al., 2013). Models for selectional preference need to ade- quately deal with the consequences of Zipf’s law: language is inherently sparse, and the majority of language utterances occur very infrequently. As a consequence, models that are based on corpus data need to properly generalize beyond the mere co-occurrence frequencies of sparse corpus data, taking into account the semantic similarity of both predicates and arguments. Researchers have come up with various approaches to this generalization step. Earlier approaches to selectional preference acquisition mostly rely on hand-crafted resources such as WordNet (Resnik, 1996; Li and Abe, 1998; Clark and Weir, 2001), while later approaches tend to take advantage of unsupervised learning machin- ery, such as latent variable models (Rooth et al., 1999; ´O S´eaghdha, 2010) and distributional simi- larity metrics (Erk, 2007; Pad´o et al., 2007). This paper investigates the use of neural net- works for the acquisition of selectional preferences. Inspired by recent advances of neural network mod- els for NLP applications (Collobert and Weston, 2008; Mikolov et al., 2013), we propose a neural network …

Content truncated to 500 words