Better Together: Modern Methods Plus Traditional Thinking in NP Alignment

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3635–3639 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3635 Better Together: Modern Methods Plus Traditional Thinking in NP Alignment ´Ad´am Kov´acs1,2, Judit ´Acs1,2, Andr´as Kornai2, G´abor Recski1,3 1BME Dept. of Automation and Applied Informatics, 2SZTAKI Institute of Computer Science, 3Apollo.AI lastname.firstname@aut.bme.hu, andras@kornai.com, gabor@apollo.ai Abstract We study a typical intermediary task to Machine Translation, the alignment of NPs in the bitext. After arguing that the task remains relevant even in an end-to-end paradigm, we present simple, dictionary- and word vector-based baselines and a BERT-based system. Our results make clear that even state of the art systems relying on the best end-to-end methods can be improved by bringing in old-fashioned methods such as stopword removal, lemmatization, and dictionaries Keywords: NP-alignment, rule-based, BERT, hybrid 1. Introduction As the state of the art in machine translation (MT) is now dominated by end-to-end (e2e) neural architectures, the task of aligning phrases of parallel corpora has been ne- glected. In fact, the tides have turned to such an extent that discovery of intermediate structure, hitherto the standard approach, now requires special justification. Since this is obviously not the place to survey the entire state of the art, we confine ourselves to a few remarks, beginning with the obvious: e2e, at least as currently practiced, is not structure free. In MT, e2e systems are built on two pivotal structures: the segmentation of the speech stream into sentences, and the segmentation of the sentences into words. As we shall see, the arguments in favor of these structural levels apply, with the same force, to the intermediary level that is our chief concern here, phrases. One simply cannot train MT sys- tems without sentence-aligned corpora (our point (1) be- low), and the embeddings that both e2e and more modular systems rely on assuming subsentential units such as words, word-parts, or characters (our (2) below). Training a truly e2e system on naturally occurring speech streams (seq2seq transformation of acoustic frames) is not on the horizon. While somewhat obscured by the fact that the segmentation effort is hidden in the preprocessing, there is no denying that e2e, as practiced today and in the foreseeable future, fundamentally relies on conventionally built structures that already pre-package a great deal of the structural informa- tion linguists take to be relevant. To be sure, characters are not exactly phonemes, and word pieces are not exactly mor- phemes, but the expectation is that the better these systems become the more the units (which in the e2e paradigm have to be teased out of the system by specialized probes) will resemble phonemes, morphemes, etc. In this paper we argue that using traditional, structure-based thinking, even about tasks such as MT that seamlessly fit the e2e paradigm, can lead to improvements in the state of the art. In the rest of this Introduction, we enumerate what we see as the main reasons for positing intermediate structures. We describe our …

Content truncated to 500 words