A Bambara Tonalization System for Word Sense Disambiguation Using Differential Coding, Segmentation and Edit Operation Filtering

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 694–703, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP A Bambara Tonalization System for Word Sense Disambiguation Using Differential Coding, Segmentation and Edit Operation Filtering Luigi (Yu-Cheng) Liu ER-TIM, INALCO, 2 rue de Lille, Paris, France luigi.liu@laposte.net Damien Nouvel ER-TIM, INALCO, 2 rue de Lille, Paris, France damien.nouvel@inalco.fr Abstract In many languages such as Bambara or Arabic, tone markers (diacritics) may be written but are actually often omitted. NLP applications are confronted to am- biguities and subsequent difficulties when processing texts. To circumvent this prob- lem, tonalization may be used, as a word sense disambiguation task, relying on con- text to add diacritics that partially disam- biguate words as well as senses. In this pa- per, we describe our implementation of a Bambara tonalizer that adds tone markers using machine learning (CRFs). To make our tool efficient, we used differential cod- ing, word segmentation and edit opera- tion filtering. We describe our approach that allows tractable machine learning and improves accuracy: our model may be learned within minutes on a 358K-word corpus and reaches 92.3% accuracy. 1 Introduction Bambara (Bamana, Bamanankan, ISO-369 Bam) is the most widely spoken language of the Mand- ing language group. It is spoken mainly in Mali (and among the considerable Malian diaspora) by 12 to 15 million people. It is not an official lan- guage; however it is the major language (besides French) on Malian radio and TV broadcasts, there are newspapers in Bambara, it is broadly used in humanities and in primary schools, and it is taught at several universities around the world. In the Mande language family, Bambara is among the best described: there are numerous works on Bambara language, such as dictionaries (Bailleul, 2007; Dumestre, 2011), description of its grammar (Dumestre, 2003; Vydrin, 1999b,a). Bambara is a tonal language, with the impor- tant drawback that its official orthography does not represent tones. Because several tonalized forms can correspond to some unaccented tokens, it makes word sense more ambiguous in corpus and imposes important challenges to NLP appli- cations. Our goal is to remedy this issue by im- plementing an automatic tonalizer for Bambara, to improve subsequent NLP processings and facili- tate linguistic analysis for Bambara. 2 Bambara Reference Corpus Our work relies on an annotated corpus, the Bam- bara Reference Corpus (BRC, in French Corpus Bambara de R´ef´erence), a linguistically annotated corpus suitable both for linguistic research and for NLP tools development. Building BRC started in 2010 as a project conducted by a small group of specialists in Manding linguistics and computer sciences in Russia. Later on, colleagues from other countries (France, Germany) joined the team to provide more texts, clean the corpus, and au- tomatically preprocess it. The corpus is available online since April 2012, most relevant information about the project may be found online1 or in pub- lications (Vydrin, 2013, 2014). Part Type Words (dist.) Punct. Non-disamb. 2,160,155 (58,277) 358,659 Disamb. 358,794 (23,875) 61,847 Table …

Content truncated to 500 words