Fully-Automatic Marker-based Chunking in 11 European Languages and Counts of the Number of Analogies between Chunks

Fully-Automatic Marker-based Chunking in 11 European Languages and Counts of the Number of Analogies between Chunks ⋆ Kota Takeya and Yves Lepage IPS, Waseda University Kitakyushu, Fukuoka 808-0135, Japan {kota-takeya@toki, yves.lepage@aoni}.waseda.jp Abstract. Analogy has been proposed as a possible principle for example-based machine translation. For such a framework to work properly, the training data should contain a large number of analogies between sentences. Consequently, such a framework can only work properly with short and repetitive sentences. To handle longer and more varied sentences, cutting the sentences into chunks could be a solution if the number of analogies between chunks is confirmed to be large. This paper thus reports counts of number of analogies using different numbers of chunk markers in 11 European languages. These experiments confirm that the number of analogies between chunks is very large: several tens of thousands of analogies between chunks extracted from sentences among which only very few analogies, if not none, were found. Keywords: Analogy, Marker Hypothesis, Marker-based Chunking, Branching Entropy 1 Introduction The example-based approach (Nagao, 1984) contrasts with the statistical approach (Brown et al., 1990; Brown et al., 1993) to machine translation in that it uses a bilingual corpus of aligned sentences as its main knowledge at run time. We aim at building an EBMT system based on proportional analogies. A translation method based on proportional analogies has been proposed by Lepage and De- noual (2005b). The following procedure gives the basic outline of the method to perform the translation of an input chunk. Let us suppose that we have a corpus of aligned chunks in two languages, German and French. Let x = “ein großes programm und” be a source chunk to be translated into one or more target chunks �x. Let the bilingual corpus consists of four chunks with their translations: ernste programme ↔ programmes s´erieux ein ernstes programm ↔ un programme s´erieux große programme und ↔ gros programmes et das ernste programm ↔ le programme s´erieux The method forms all possible analogies with all possible triples of chunks from the parallel cor- pus. Among them: ernste programme : ein ernstes programm :: große programme und : ein großes programm und = x ⋆ This paper is a part of the outcome of research performed under a Waseda University Grant for Special Research Project (project number: 2010A-906). Copyright 2011 by Kota Takeya and Yves Lepage 25th Pacific Asia Conference on Language, Information and Computation, pages 567–576 567 Taking into account the knowledge in the bilingual corpus, analogical equation can be formed in the target language: programmes s´erieux : un programme s´erieux :: gros programmes et : �x Its solution is a candidate translation of the source chunk: �x = “un gros programme et” For such an EBMT system to work well, the more numerous the proportional analogies, the better the translation outputs are expected to be. The method can work on small sentences like the ones in the BTEC corpus (Lepage and Denoual, 2005a), but cannot handle long sentences like the …

Content truncated to 500 words