Fast Semantic Extraction Using a Novel Neural Network Architecture

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 560–567, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Fast Semantic Extraction Using a Novel Neural Network Architecture Ronan Collobert NEC Laboratories America, Inc. 4 Independence Way Suite 200, Princeton, NJ 08540 collober@nec-labs.com Jason Weston NEC Laboratories America, Inc. 4 Independence Way Suite 200, Princeton, NJ 08540 jasonw@nec-labs.com Abstract We describe a novel neural network archi- tecture for the problem of semantic role la- beling. Many current solutions are compli- cated, consist of several stages and hand- built features, and are too slow to be applied as part of real applications that require such semantic labels, partly because of their use of a syntactic parser (Pradhan et al., 2004; Gildea and Jurafsky, 2002). Our method in- stead learns a direct mapping from source sentence to semantic tags for a given pred- icate without the aid of a parser or a chun- ker. Our resulting system obtains accuracies comparable to the current state-of-the-art at a fraction of the computational cost. 1 Introduction Semantic understanding plays an important role in many end-user applications involving text: for infor- mation extraction, web-crawling systems, question and answer based systems, as well as machine trans- lation, summarization and search. Such applications typically have to be computationally cheap to deal with an enormous quantity of data, e.g. web-based systems process large numbers of documents, whilst interactive human-machine applications require al- most instant response. Another issue is the cost of producing labeled training data required for statisti- cal models, which is exacerbated when those models also depend on syntactic features which must them- selves be learnt. To achieve the goal of semantic understanding, the current consensus is to divide and conquer the [The company]ARG0 [bought]REL [sugar]ARG1 [on the world market]ARGM-LOC [to meet export commitments]ARGM-PNC Figure 1: Example of Semantic Role Labeling from the PropBank dataset (Palmer et al., 2005). ARG0 is typically an actor, REL an action, ARG1 an ob- ject, and ARGM describe various modifiers such as location (LOC) and purpose (PNC). problem. Researchers tackle several layers of pro- cessing tasks ranging from the syntactic, such as part-of-speech labeling and parsing, to the semantic: word-sense disambiguation, semantic role-labeling, named entity extraction, co-reference resolution and entailment. None of these tasks are end goals in themselves but can be seen as layers of feature ex- traction that can help in a language-based end ap- plication, such as the ones described above. Un- fortunately, the state-of-the-art solutions of many of these tasks are simply too slow to be used in the ap- plications previously described. For example, state- of-the-art syntactic parsers theoretically have cubic complexity in the sentence length (Younger, 1967)1 and several semantic extraction algorithms use the parse tree as an initial feature. In this work, we describe a novel type of neural network architecture that could help to solve some of these issues. We focus our experimental study on the semantic role labeling problem (Palmer et al., 2005): being able to give a …

Content truncated to 500 words