Combining a self-organising map with memory-based learning

Combining a self-organising map with memory-based learning James Hammerton james.hammerton@ucd.ie Dept of Computer Science University College Dublin Ireland Erik Tjong Kim Sang erikt@uia.ua.ac.be CNTS – Language Technology Group University of Antwerp Belgium Abstract Memory-based learning (MBL) has enjoyed considerable success corpus-based natural language process-in ing (NLP) tasks and is thus a reliable method of getting a high-level of per- formance when building corpus-based NLP systems. However there is a bottleneck in MBL whereby any novel testing item has to be compared against all the training items in memory base. For this reason there has been some interest in various forms of memory editing whereby some method of selecting a subset of the memory base is employed to reduce the number of comparisons. This paper investigates the use of a modified self-organising map (SOM) to select a subset of the memory items for comparison. This method involves reducing the number of comparisons to a value proportional to the square root of the number of training items. The method is tested on the identification of base noun-phrases in the Wall Street Journal corpus, using sections 15 to 18 for training and section 20 for testing. 1 Introduction Currently, there is considerable interest in ma- chine learning methods for corpus-based lan- guage learning. A promising technique here is memory-based learning1 (MBL) (Daelemans et al., 1999a), where a task is redescribed as a classi- fication problem. The classification is performed by matching an input item to the most similar of a set of training items and choosing the most fre- quent classification of the closest item(s). Sim- ilarity is computed using an explicit similarity metric. MBL performs well by bringing all the training data to bear on the task. This is done at the cost, in the worst case, of comparing novel items to all of the training items to find the closest match. There is thus some interest in developing memory edit- ing techniques to select a subset of the items for comparison. This paper investigates whether a self- organising map (SOM) can be used to perform memory editing without reducing performance. The system is tested on base noun-phrase (NP) chunking using the Wall Street Journal corpus (Marcus et al., 1993). 2 The Self-Organising Map (SOM) The SOM was developed originally by Kohonen (1990) and has found a wide range of uses from classification to storing a lexicon. It operates as follows (see Figure 1). The SOM consists of two layers, an input layer and the map. Each unit in the map has a vector of weights associated with it that is the same size as that of the input layer. When an input is presented to the SOM, the unit whose weight vector is closest to the input vector is selected as a winner. 1Also known as instance based learning. INPUTS MAP Figure 1: The Self-Organising Map. The map units respond to the inputs. The map unit whose weight vector is closest to the input vector be- comes the winner. During training, …

Content truncated to 500 words