How can machine translation help generate Arab melodic improvisation?

How can machine translation help generate Arab melodic improvisation? Fadi Al-Ghawanmeh Music Dept., Univ. of Jordan SMarT Group, LORIA, F-54600 RITMO Centre, Univ. of Oslo fadi.al-ghawanmeh @loria.fr Alexander Refsum Jensenius RITMO Centre Department of Musicology University of Oslo, Norway a.r.jensenius @imv.uio.no Kamel Smaili SMarT Group LORIA, F-54600 University of Lorraine, France smaili@loria.fr Abstract This article presents a system to generate Arab music improvisation using machine translation (MT). To reach this goal, we developed a MT model to translate a vo- cal improvisation into an automatic instru- mental oud (Arab lute) response. Given the melodic and non-metric musical form, it was necessary to develop efficient tex- tual representations in order for classical MT models to be as successful as in com- mon NLP applications. We experimented with Statistical and Neural MT to train our parallel corpus (Vocal → Instrument) of 6991 sentences. The best model was then used to generate improvisation by it- eratively translating the translations of the most common patterns of each maq¯am (n-grams), producing elaborated variations conditioned to listener feedback. We con- structed a dataset of 717 instrumental im- provisations to extract their n-grams. Ob- jective evaluation of MT was conducted at two levels: a sentence-level evaluation us- ing the BLEU metric, and a higher level evaluation using musically informed met- rics. Objective measures were consistent with one another. Subjective evaluations by experts from the maq¯am music tradition were promising, and a useful reference for understanding objective results. © 2023 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. Figure 1: The most common maq¯am¯ats in Arab Music (Al- Abbas, 1986). 1 Introduction The purpose of this paper is to present a method for using machine translation (MT) to gener- ate automatic instrumental improvisation in Arab maq¯am music, particularly in two contexts: re- sponsive accompaniment to vocal improvisation (maww¯al), and free instrumental improvisation (taq¯as¯ım). This method could then be adapted to other melodic musical traditions. We situate this project within the efforts to maintain, preserve, and develop these musical forms in Arab music using (MT) paradigms. We construct our own corpora and tools for their collection and processing, ex- plore neural and statistical methods, and test them using the bilingual evaluation understudy (BLEU) measure, which is a common MT metric (Papineni et al., 2002). The study presents the results of us- ing a BLEU score along with musically informed metrics (Yang and Lerch, 2020) and subjective evaluations. More broadly, we view this music- based project as a MT challenge within the broader context of under-resourced languages (Krauwer, 2003) (Berment, 2004). In Arab music, maww¯al is a non-metric vocal improvisation and is often applied to narrative po- etry. Upon the completion of each vocal sentence, the instrumentalist performs a recapitulation, or a translation of that sentence (Racy, 1998) (Farraj, 2007). In other words, the duo (singer, instru- mentalist) play a musical conversation. At time ti, the singer produces an improvisatory phrase p(ti), then the instrumentalist produces a musical answer corresponding …

Content truncated to 500 words