No Data to Crawl? Monolingual Corpus Creation from PDF Files of Truly low-Resource Languages in Peru

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2914–2923 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2914 No data to crawl? Monolingual corpus creation from PDF files of truly low-resource languages in Peru Gina Bustamante♠, Arturo Oncevay♠♢ and Roberto Zariquiey♣ ♠Artificial Intelligence Research Group, Pontificia Universidad Cat´olica del Per´u ♢School of Informatics, University of Edinburgh ♣Department of Humanities, Linguistics Unit, Pontificia Universidad Cat´olica del Per´u {gina.bustamante,arturo.oncevay,rzariquiey}@pucp.edu.pe Abstract We introduce new monolingual corpora for four indigenous and endangered languages from Peru: Shipibo-konibo, Ashaninka, Yanesha and Yine. Given the total absence of these languages in the web, the extraction and processing of texts from PDF files is relevant in a truly low-resource language scenario. Our procedure for monolingual corpus creation considers language-specific and language-agnostic steps, and focuses on educational PDF files with multilingual sentences, noisy pages and low-structured content. Through an evaluation based on language modelling and character-level perplexity on a subset of manually extracted sentences, we determine that our method allows the creation of clean corpora for the four languages, a key resource for natural language processing tasks nowadays. Keywords: Shipibo-Konibo, Ashaninka, Yanesha, Yine, endangered languages, indigenous languages, low-resource languages, pdf processing, monolingual corpus, corpus creation 1. Introduction Several natural language processing (NLP) tasks are cur- rently improving their performance based on deep learn- ing models, which require large amounts of data to learn from. For this reason, unlabelled text corpora have become an elemental resource to improve performance in various NLP applications, such as in representation learning (De- vlin et al., 2019), language modelling (Buck et al., 2014), and neural machine translation with back-translation (Sen- nrich et al., 2016) or in an unsupervised scenario (Artetxe et al., 2018; Lample et al., 2018). Other non-language gener- ation tasks, such as text classification, can also benefit in a transfer learning setting, as the model can focus on the spe- cific goal and not in learning the representation of words or subwords (Howard and Ruder, 2018). Monolingual data is usually inexpensive to obtain. A pop- ular and widely method is crawling different web pages. However, this is only possible under the assumption that there are sufficient web sites written in the target language. This is not the case for many endangered languages that do not have a consistent presence in the World Wide Web. In this way, the lack of web content limits the creation of lan- guage resources and developing further NLP applications for truly low-resource languages. The aforementioned scenario is common for most of the indigenous and endangered languages in Peru, a multicul- tural and multilingual country. There is almost not web content of the Peruvian native languages, even in official Government sites. Nonetheless, recent efforts in bilingual education (Spanish – native language) for indigenous com- munities have endorsed the creation of digital educational resources written in local languages. These documents are stored in online repositories, mostly in Portable Document Format (PDF). We could take advantage of the available PDF …

Content truncated to 500 words