Quantifying Synthesis and Fusion and their Impact on Machine Translation

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1308 - 1321 July 10-15, 2022 ©2022 Association for Computational Linguistics Quantifying Synthesis and Fusion and their Impact on Machine Translation Arturo Oncevayϵ∗ Duygu Atamanη Niels van Berkelα,µ Barry Haddowϵ Alexandra Birchϵ Johannes Bjervaα,σ ϵUniversity of Edinburgh ηNew York University αAalborg University (Aalborgµ | Copenhagenσ) a.oncevay@ed.ac.uk Abstract Theoretical work in morphological typology offers the possibility of measuring morpholog- ical diversity on a continuous scale. How- ever, literature in Natural Language Process- ing (NLP) typically labels a whole language with a strict type of morphology, e.g. fu- sional or agglutinative. In this work, we pro- pose to reduce the rigidity of such claims, by quantifying morphological typology at the word and segment level. We consider Payne (2017)’s approach to classify morphology us- ing two indices: synthesis (e.g. analytic to polysynthetic) and fusion (agglutinative to fu- sional). For computing synthesis, we test un- supervised and supervised morphological seg- mentation methods for English, German and Turkish, whereas for fusion, we propose a semi-automatic method using Spanish as a case study. Then, we analyse the relationship between machine translation quality and the degree of synthesis and fusion at word (nouns and verbs for English-Turkish, and verbs in English-Spanish) and segment level (previous language pairs plus English-German in both di- rections). We complement the word-level anal- ysis with human evaluation, and overall, we observe a consistent impact of both indexes on machine translation quality. 1 Introduction One of the first barriers to develop language tech- nologies is morphology, i.e., how systematically diverse their word formation processes are. For in- stance, agglutination and fusion are two morpholog- ical kind of processes that concatenate morphemes to a root with explicit or non-explicit boundaries, respectively. Processing morphologically-diverse languages and evaluating morphological compe- tence in NLP models is relevant for language gen- eration and understanding tasks, such as machine translation (MT). It is unfeasible to develop models ∗Work started when the first author was doing a research internship with JB at Aalborg University, Campus Copenhagen with capacity large enough to encode the full vocab- ulary of every language, and it is a must to rely on subword segmentation approaches that help to con- strain the capacity when generating rare, or even new words (Sennrich et al., 2016). Hence, under- standing morphology is essential to develop robust subword-based models and evaluate the quality of their outputs (Vania and Lopez, 2017). Neverthe- less, there is a potential gap between the probing of whether an NLP model can handle "morpho- logical richness", and what is a proper measure of "morphological richness" from linguistic typology. In most of the recent NLP literature, different types of languages (e.g. agglutinative, polysyn- thetic) are chosen to test a more diverse handling of morphological richness (Ponti et al., 2019). There is, however, a debate as to whether languages can indeed be classified into discrete morphological categories. Payne (2017) provided a morphological typology measurement in a continuous spectrum …

Content truncated to 500 words