Exploring a Choctaw Language Corpus with Word Vectors and Minimum Distance Length

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2746–2753 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2746 Exploring a Choctaw Language Corpus with Word Vectors and Minimum Distance Length Jacqueline Brixey1 3, David Sides2, Timothy Vizthum, David Traum3, Khalil Iskarous1 1University of Southern California, 2Mississippi State University, 3USC Institute for Creative Technologies brixey@usc.edu, dhs131@msstate.edu, t.vizthum@gmail.com, traum@ict.usc.edu, kiskarou@usc.edu Abstract This work introduces additions to the corpus ChoCo (Choctaw language Corpus), a multimodal corpus for the American indigenous language Choctaw. Using texts from the corpus, we develop new computational resources by using two off-the-shelf tools: word2vec and Linguistica. Our results indicate these tools need expert input to reliably interpret the results. Keywords: endangered languages, indigenous language, low resource languages, Choctaw, Native American languages 1. Introduction Developing computational resources for a language is a time and labor-intensive undertaking. Many languages in the world are becoming endangered and extinct before re- sources can be developed. However, several off-the-shelf tools exist that can be used to explore low resource lan- guages that require no expert knowledge nor substantial la- bor. In this work, we use two such tools to create new computa- tional resources for Choctaw, an American indigenous lan- guage. The first tool is word2vec, which provides lexical understanding of a language without the use of a dictionary or word list. The second tool, Linguistica, utilizes unsuper- vised learning to detect morphological information. Our experiments demonstrate that off-the-shelf tools could be used with minimal labor to increase available linguis- tic resources for low-resource languages. However, expert knowledge was still needed to interpret the results and re- move numerous false positives. We suggest how to tune two tools to return more meaningful results for similar lan- guages, thus creating a road map for increasing resources for other under-resourced indigenous languages. Our other contribution in this paper is to introduce new ad- ditions to our previous corpus work for Choctaw (Brixey et al., 2018). We added over 90,000 new tokens of text from diverse sources, as well as an additional dictionary. The corpus contains a diversity of text, audio, and video. 2. Choctaw People and Language The Choctaw language is spoken by the Choctaw tribe, an American indigenous group that originally lived in what to- day is Alabama and Mississippi. In the early 1830s the Choctaws were forcibly relocated to Oklahoma in the mi- gration known as the Trail of Tears, although some people remained. Choctaws are the third most populous tribe by tribal group population in the United States, with around 195,000 peo- ple identifying as Choctaw in the 2010 US census.1 There 1https://www.census.gov/population/www/ cen2010/cph-t/t-6tables/TABLE%20(1).pdf are three federally recognized Choctaw tribes2: Jena Band of Choctaw Indians (in Louisiana), Mississippi Band of Choctaw Indians, and the Choctaw Nation of Oklahoma. The first and third authors are both enrolled members of the Choctaw Nation of Oklahoma. The Choctaw language has a “Threatened” classification according to Ethnologue (Simons and Fennig, 2018), as there are only approximately 10,000 …

Content truncated to 500 words