KC4Align: Improving Sentence Alignment method for Low-resource Language Pairs

KC4Align: Improving Sentence Alignment Method for Low-resource Language Pairs Ha Nguyen Tien Multilingual Translation Project KC4.0 VNU - UET, Hanoi, Vietnam tienhapt@gmail.com Huong Le Thanh Hanoi University of Science and Technology Hanoi, Vietnam huonglt@soict.hust.edu.vn Dat Nguyen Huu Multilingual Translation Project KC4.0 VNU - UET, Hanoi, Vietnam cs.nguyenhuudat@gmail.com, Vinh Nguyen Van and Minh Nguyen Quang Multilingual Translation Project KC4.0 VNU - UET, Hanoi, Vietnam vinhnv2000@gmail.com and nqminh@vnu.edu.vn Abstract Bilingual corpus is an important resource for many applications of natural language pro- cessing (NLP). About low-resource language pairs, it is more necessary to build them au- tomatically. In this paper, we propose a novel method that uses an embedding model with a sentence length ratio to align sentences from bilingual documents for low-resource language pairs. Our method is inspired by Ve- calign, the state-of-the-art method, and over- comes its limitation. Experiments on the low- resource Vietnamese-Lao language pair show that our proposed method achieves higher pre- cision and recall than other good methods even on noisy data. 1 Introduction Low-resource languages (LRLs)can be understood as less studied, resource scarce, less computerized, less privileged, less commonly taught, or low den- sity, among other denominations (Magueresse et al., 2020), such as Lao, Khmer, Urdu, etc. It is diffi- cult to build large datasets of such languages for the training task. Because of that, systems that apply machine learning or deep learning approaches often give low results or fail to work with such languages. Machine translation has achieved significant im- provement due to the application of deep learning models. This approach only works well with a large parallel corpus for training the model. To create such a corpus, we need good resources of parallel text, as well as a mechanism to align sentences in bilingual text pairs. The sentence alignment can be done by creating all candidate sentence pairs from bilingual docu- ments and then computing the semantic similarity between these sentence pairs. The simplest method to measure their semantic similarity is to count the common words between the two documents using bilingual dictionary. Some researches translate the two documents to a third language which is a rich- resources and popular one such as English, then use a similarity score such as Levenstein or co- sine to compute their similarity (Abdul-Rauf and Schwenk, 2009) (Sennrich and Volk, 2011). Recent works (Grégoire and Langlais, 2018) (Artetxe and Schwenk, 2019a) (Thompson and Koehn, 2019a) (Chousa et al., 2020) apply a multi-lingual language model such as BERT or LASER to represent their sentence embedding vectors and compute the co- sine similarity between these vectors. However, this method only provides good results for the language pairs that have enough training data for the language model. This method cannot apply to the language pairs which one of them is not supported by the mul- tilingual language model. Vietnamese Laos is one of such language pairs. There is not any language em- bedding model support Laos language. LRLs also suffer from this problem. As far as we know, there is no …

Content truncated to 500 words