Automatic Generation of Parallel Treebanks

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 1105–1112 Manchester, August 2008 Automatic Generation of Parallel Treebanks Ventsislav Zhechev NCLT, School of Computing Dublin City University Dublin, Ireland vzhechev@computing.dcu.ie Andy Way NCLT, School of Computing Dublin City University Dublin, Ireland away@computing.dcu.ie Abstract The need for syntactically annotated data for use in natural language processing has increased dramatically in recent years. This is true especially for parallel treebanks, of which very few exist. The ones that exist are mainly hand-crafted and too small for reliable use in data-oriented applications. In this paper we introduce a novel platform for fast and robust automatic generation of parallel treebanks. The software we have developed based on this platform has been shown to handle large data sets. We also present evaluation results demonstrating the quality of the derived treebanks and discuss some possible modifications and improvements that can lead to even better results. We expect the presented platform to help boost research in the field of data- oriented machine translation and lead to advancements in other fields where paral- lel treebanks can be employed. 1 Introduction In recent years much effort has been made to make use of syntactic information in statistical machine translation (MT) systems (Hearne and Way, 2006, Nesson et al., 2006). This has led to increased in- terest in the development of parallel treebanks as the source for such syntactic data. They consist of a parallel corpus, both sides of which have been parsed and aligned at the sub-tree level. So far parallel treebanks have been created manually or semi-automatically. This has proven to be a laborious and time-consuming task that is prone to errors and inconsistencies (Samuelsson and Volk, 2007). Because of this, only a few paral- lel treebanks exist and none are of sufficient size for productive use in any statistical MT application. In this paper we present a novel platform for the automatic generation of parallel treebanks from parallel corpora and discuss several meth- ods for the evaluation of the results. We discuss algorithms both for cases in which monolingual parsers exist for both languages and for cases in which such parsers are not available. The parallel treebanks created with the methods described in this paper can be used by different statistical MT applications and for translation studies. We start in section 2 by introducing the tech- niques for automatic generation of parallel tree- banks. The evaluation methods and results are introduced in section 3 and in section 4 we give suggestions for possible improvements to the generation technology and to the evaluation algo- rithms. Finally, in section 5 we present existing parallel treebanks and conclude in section 6. 2 Automatic Generation of Parallel Treebanks In this section we introduce a method for the auto- matic generation of parallel treebanks from paral- lel corpora. The only tool that is required besides the software presented in this paper is a word alignment tool. Such tools exist and some are freely available (eg. GIZA++ (Och and Ney, …

Content truncated to 500 words