A Connectionist Architecture for Learning to Parse

A Connectionist Architecture for Learning to Parse James Henderson and Peter Lane Dept of Computer Science, Univ of Exeter Exeter EX4 4PT, United Kingdom j amie~dcs, ex. ac. uk, pclane©dcs, ex. ac. uk Abstract We present a connectionist architecture and demon- strate that it can learn syntactic parsing from a cor- pus of parsed text. The architecture can represent syntactic constituents, and can learn generalizations over syntactic constituents, thereby addressing the sparse data problems of previous connectionist ar- chitectures. We apply these Simple Synchrony Net- works to mapping sequences of word tags to parse trees. After training on parsed samples of the Brown Corpus, the networks achieve precision and recall on constituents that approaches that of statistical methods for this task. 1 Introduction Connectionist networks are popular for many of the same reasons as statistical techniques. They are ro- bust and have effective learning algorithms. They also have the advantage of learning their own inter- nal representations, so they are less constrained by the way the system designer formulates the prob- lem. These properties and their prevalence in cog- nitive modeling has generated significant interest in the application of connectionist networks to natu- ral language processing. However the results have been disappointing, being limited to artificial do- mains and oversimplified subproblems (e.g. (Elman, 1991)). Many have argued that these kinds of con- ncctionist networks are simply not computationally adequate for learning the complexities of real natural language (e.g. (Fodor and Pylyshyn, 1988), (Hender- son, 1996)). Work on extending connectionist architectures for application to complex domains such as natural lan- guage syntax has developed a theoretically moti- vated technique called Temporal Synchrony Variable Binding (Shastri and Ajjanagadde, 1993; Henderson, 1996). TSVB allows syntactic constituency to be represented, but to date there has been no empiricM demonstration of how a learning algorithm can be effectively applied to such a network. In this paper we propose an architecture for TSVB networks and empirically demonstrate its ability to learn syntac- tic parsing, producing results approaching current statistical techniques. In the next section of this paper we present the proposed connectionist architecture, Simple Sym chrony Networks (SSNs). SSNs are a natural ex- tension of Simple Recurrent Networks (SRNs) (El- man, 1991), which are in turn a natural extension of Multi-Layered Perceptrons (MLPs) (Rumelhart et al., 1986). SRNs are an improvement over MLPs because they generalize what they have learned over words in different sentence positions. SSNs are an improvement over SRNs because the use of TSVB gives them the additional ability to generalize over constituents in different structural positions. The combination of these generalization abilities is what makes SSNs adequate for syntactic parsing. Section 3 presents experiments demonstrating SSNs' ability to learn syntactic parsing. The task is to map a sentence's sequence of part of speech tags to either an unlabeled or labeled parse tree, as given in a preparsed sample of the Brown Cor- pus. A network input-output format is developed for this task, along with some linguistic assump- tions that were used to …

Content truncated to 500 words