Does Sentence Segmentation Matter for Machine Translation?

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 843 - 854 December 7-8, 2022 ©2022 Association for Computational Linguistics Does Sentence Segmentation Matter for Machine Translation? Rachel Wicks1 and Matt Post1,2 1Center for Language and Speech Processing 2Human Language Technology Center of Excellence Johns Hopkins University {rewicks@,post@cs.}jhu.edu Abstract For the most part, NLP applications operate at the sentence level. Since sentences occur most naturally not on their own but embedded in doc- uments, they must be extracted and segmented via the use of a segmenter, of which there are a handful of options. There has been some work evaluating the performance of segmenters on intrinsic metrics, that look at their ability to re- cover human-segmented sentence boundaries, but there has been no work looking at the ef- fect of segmenters on downstream tasks. We ask the question, “does segmentation matter?" and attempt to answer it on the task of machine translation. We consider two settings: the in- ference scenario, where sentences are passed into a black-box system whose training seg- mentation is mostly unknown, and the train- ing setting, where researchers have full control over the process. We find that the choice of segmenter largely does not matter, so long as its behavior is not one of extreme under- or over-segmentation. For such settings, we pro- vide some qualitative analysis examining their harms, and point the way towards document- level processing. 1 Introduction Contemporary machine translation assumes a sentence-level paradigm. However, data doesn’t exist naturally at the sentence level, requiring the use of automatic segmenters to split the data at both training and inference time. Training data is prepared with the use of sentence segmenters,1 which are preprocessing steps that occur prior to alignment and bitext creation. At test time, de- ployed models also require the use of a segmenter. Many times, for downloaded models, especially, this inference-time application must be made with- out knowing what segmenter was used to train the model, introducing a potential misalignment or dis- crepancy and resulting performance degradation. 1Sometimes called sentence breakers. Sentence segmentation itself has received only a little attention in the research literature, although there has been a recent uptick (Moore, 2021; Wicks and Post, 2021). But to our knowledge, no work has been done investigating the effects of segmenta- tion on machine translation. In fact, most research papers do not deal with the question at all, rely- ing as they do on pre-segmented parallel data for both training and test time. This is a practical prob- lem for deployment scenarios, where segmentation must be considered. It is also a deeper problem, since segmentation is ultimately a modeling deci- sion that should be noted and made available with any published models, such as is done for other modeling decisions affecting input text, such as normalization, tokenization, and subword process- ing. To understand whether and to what extent seg- mentation matters, we ask a series of questions: (1) What segmenter is best used at inference time? (2) When training a model, …

Content truncated to 500 words