Manual vs Automatic Bitext Extraction

Manual vs Automatic Bitext Extraction Aibek Makazhanov, Bagdat Myrzakhmetov, Zhenisbek Assylbekov Nazarbayev University, National Laboratory Astana Nazarbayev University, School of Science and Technology 53 Kabanbay Batyr ave., Astana, Kazakhstan aibek.makazhanov@nu.edu.kz, bagdat.myrzakhmetov@nu.edu.kz, zhassylbekov@nu.edu.kz Abstract We compare manual and automatic approaches to the problem of extracting bitexts from the Web in the framework of a case study on building a Russian-Kazakh parallel corpus. Our findings suggest that targeted, site-specific crawling results in cleaner bitexts with a higher ratio of parallel sentences. We also find that general crawlers combined with boilerplate removal tools tend to retrieve shorter texts, as some content gets cleaned out with the markup. When it comes to sentence splitting and alignment we show that investing some effort in data pre- and post-processing as well as fiddling with off-the-shelf solutions pays a noticeable dividend. Overall we observe that, depending on the source, automatic bitext extraction methods may lack severely in coverage (retrieve fewer sentence pairs) and on average are fewer precise (retrieve less parallel sentence pairs). We conclude that if one aims at extracting high-quality bitexts for a small number of language pairs, automatic methods best be avoided, or at least used with caution. Keywords: bitext extraction, crawling, sentence alignment 1. Introduction The Web has long been used as one of, if not, the most important source for obtaining language resources of various purpose, size and quality. Given the large amount and accessibility of text available in a vari- ety of languages, it is particularly appealing to mine the Web for parallel and comparable corpora (bitexts), resources essential for building and evaluating data- driven machine translation systems. The task of extracting bitexts from the Web typically involves the following four major consecutive steps: (i) data collection (crawling), (ii) document alignment, (iii) sentence segmentation, and (iv) sentence align- ment. Depending on the degree of user involvement in the first two steps, approaches to the problem can be classified roughly as: (i) manual methods (Koehn, 2005; Tiedemann, 2007; Myrzakhmetov et al., 2016) that crawl specific websites and usually rely on their id- iosyncratic properties at the document alignment step; (ii) semi-automatic methods (Esplà-Gomis and For- cada, 2010; Papavassiliou et al., 2013) that make no site-specific assumptions, but still require a target list of URLs to extract bitexts from; (iii) completely au- tomatic methods (Smith et al., 2013; Ljubes̆ić et al., 2016; Resnik and Smith, 2003) that crawl the entire Web and produce whatever bitexts they can. Choosing appropriate extraction strategy depends greatly on the task at hand. Thus, if the goal is to extract bitexts for dozens of language pairs automatic approaches may come in handy. However, if one needs a high quality parallel corpus for just one or two pairs of languages, what is the difference between bitexts obtained by manual and automatic methods? Is the final quality of manual extraction worth time and ef- fort? To our disappointment, we could not find de- tailed answers to these questions in the literature. Var- ious implementations of the extraction strategies are typically …

Content truncated to 500 words