Unsupervised Morphology-Based Vocabulary Expansion

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 1349–1359, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Unsupervised Morphology-Based Vocabulary Expansion Mohammad Sadegh Rasooli, Thomas Lippincott, Nizar Habash and Owen Rambow Center for Computational Learning Systems Columbia University, New York, NY, USA {rasooli,tom,habash,rambow}@ccls.columbia.edu Abstract We present a novel way of generating un- seen words, which is useful for certain ap- plications such as automatic speech recog- nition or optical character recognition in low-resource languages. We test our vo- cabulary generator on seven low-resource languages by measuring the decrease in out-of-vocabulary word rate on a held-out test set. The languages we study have very different morphological properties; we show how our results differ depend- ing on the morphological complexity of the language. In our best result (on As- samese), our approach can predict 29% of the token-based out-of-vocabulary with a small amount of unlabeled training data. 1 Introduction In many applications in human language technolo- gies (HLT), the goal is to generate text in a target language, using its standard orthography. Typical examples include automatic speech recognition (ASR, also known as STT or speech-to-text), opti- cal character recognition (OCR), or machine trans- lation (MT) into a target language. We will call such HLT applications “target-language genera- tion technologies” (TLGT). The best-performing systems for these applications today rely on train- ing on large amounts of data: in the case of ASR, the data is aligned audio and transcription, plus large unannotated data for the language model- ing; in the case of OCR, it is transcribed optical data; in the case of MT, it is aligned bitexts. More data provides for better results. For languages with rich resources, such as English, more data is often the best solution, since the required data is readily available (including bitexts), and the cost of anno- tating (e.g., transcribing) data is outweighed by the potential significance of the systems that the data will enable. Thus, in HLT, improvements in qual- ity are often brought about by using larger data sets (Banko and Brill, 2001). When we move to low-resource languages, the solution of simply using more data becomes less appealing. Unannotated data is less readily avail- able: for example, at the time of publishing this paper, 55% of all websites are in English, the top 10 languages collectively account for 90% of web presence, and the top 36 languages have a web presence that covers at least 0.1% of web sites.1 All other languages (and all languages considered in this paper except Persian) have a web presence of less than 0.1%. Considering Wikipedia, another resource often used in HLT, English has 4.4 mil- lion articles, while only 48 other languages have more than 100,000.2 As attention turns to de- veloping HLT for more languages, including low- resource languages, alternatives to “more-data” approaches become important. At the same time, it is often not possible to use knowledge-rich approaches. For low-resource lan- guages, resources such as morphological analyz- ers are not …

Content truncated to 500 words