Development of Natural Language Processing Tools for Cook Islands Māori

Development of Natural Language Processing Tools for Cook Islands M¯aori Rolando Coto-Solano1, Sally Akevai Nicholas2, and Samantha Wray3 1School of Linguistics and Applied Language Studies Victoria University of Wellington Te Whare W¯ananga o te ¯Upoko o te Ika a M¯aui rolando.coto@vuw.ac.nz 2School of Language and Culture Auckland University of Technology sally.nicholas@aut.ac.nz 3Neuroscience of Language Lab New York University Abu Dhabi, United Arab Emirates samantha.wray@nyu.edu Abstract This paper presents three ongoing projects for NLP in Cook Islands M¯aori: Un- trained Forced Alignment (approx. 9% er- ror when detecting the center of words), automatic speech recognition (37% WER in the best trained models) and automatic part-of-speech tagging (92% accuracy for the best performing model). These new re- sources fill existing gaps in NLP for the language, including gold standard POS- tagged written corpora, transcribed speech corpora, and time-aligned corpora down to the phoneme level. These are part of efforts to accelerate the documentation of Cook Islands M¯aori and to increase its vi- tality amongst its users. 1 Introduction Cook Islands M¯aori has been moderately well documented with two dictionaries (Buse et al., 1996; Savage, 1962), a comprehensive description (Nicholas, 2017), and a corpus of audiovisual ma- terials (Nicholas, 2012). However these materials are not yet sufficiently organized or annotated so as to be machine readable and thus maximally use- ful for both scholarship and revitalization projects. The human resources needed to achieve the de- sired level of annotation are not available, which has encouraged us to take advantage of NLP meth- ods to accelerate documentation and research. 1.1 Minority Languages and NLP Lack of resources makes it difficult to train data- driven NLP tools for smaller languages. This is compounded by the difficulty in generating input for Indigenous and endangered languages, where dwindling numbers of speakers, non-standardized writing systems and lack of resources to train spe- cialist transcribers and analysts create a vicious cycle that makes it even more difficult to take ad- vantage of NLP solutions. Amongst the hundreds of languages of the Americas, for example, very few have large spoken and written corpora (e.g. Zapotec from Mexico, Guaran´ı from Paraguay and Quechua from Bolivia and Per´u), some have spo- ken and written corpora, and only a handful pos- sess more sophisticated tools like spell-checkers and machine translation (Mager et al., 2018). Overcoming these limitations is an important part of accelerating language documentation. Cre- ating NLP resources also enhances the profile of endangered languages, creating a symbolic impact to generate positive associations towards the lan- guage and attract new learners, particularly young members of the community who might not other- wise see their language in a digital environment (Aguilar Gil, 2014; Kornai, 2013). As for Polynesian languages, te reo M¯aori, the Indigenous language spoken in Aotearoa New Zealand, is the one that has received the most at- tention from the NLP community. It has multiple corpora and Google provides machine-translations for it as part of Google Translate, and tools such as speech-to-text, text-to-speech and parsing are Rolando Coto Solano, Sally Akevai …

Content truncated to 500 words