Findings of the WMT 2021 Biomedical Translation Shared Task: Summaries of Animal Experiments as New Test Set

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 664–683 November 10–11, 2021. ©2021 Association for Computational Linguistics 664 Findings of the WMT 2021 Biomedical Translation Shared Task: Summaries of Animal Experiments as New Test Set Lana Yeganova1∗ Dina Wiemann2 Mariana Neves3 Federica Vezzani4 Amy Siu5 Iñigo Jauregi Unanue6 Maite Oronoz7 Nancy Mah8 Aurélie Névéol9 David Martinez10,11 Rachel Bawden12 Giorgio Maria Di Nunzio13 Roland Roller14 Philippe Thomas14 Cristian Grozea15 Olatz Perez de Viñaspre7 Maika Vicente Navarro16 Antonio Jimeno Yepes10 1NCBI/NLM/NIH, Bethesda, USA 2Novartis AG, Basel, Switzerland 3German Centre for the Protection of Laboratory Animals (Bf3R), German Federal Institute for Risk Assessment (BfR), Berlin, Germany 4Dept. of Linguistic and Literary Studies University of Padua, Italy 5Berliner Hochschule für Technik, Germany 6University of Technology Sydney, Sydney, Australia 7IXA NLP Group, University of the Basque Country, Donostia, Spain 8Fraunhofer Institute for Biomedical Engineering (IBMT), Berlin, Germany 9LISN, CNRS, Université Paris-Saclay, Orsay, France 10University of Melbourne, Australia 11Doctor Evidence, Santa Monica, CA, USA 12Inria, France 13Dept. of Information Engineering, University of Padua, Italy 14German Research Center for Artificial Intelligence (DFKI), Berlin, Germany 15Fraunhofer Institute FOKUS, Berlin, Germany 16Maika Spanish Translator, Melbourne, Australia Abstract In the sixth edition of the WMT Biomed- ical Task, we addressed a total of eight language pairs, namely English/German, English/French, English/Spanish, En- glish/Portuguese, English/Chinese, En- glish/Russian, English/Italian, and En- glish/Basque. Further, our tests were composed of three types of textual test sets. New to this year, we released a test set of summaries of animal experiments, in addition to the test sets of scientific abstracts and terminologies. We received a total of 107 ∗ The organization of the biomedical task is complex and relies on varied essential contributions from many individuals. Authors are listed randomly because we could not do justice to the contributors using a single ranking. We would like to acknowledge MN for dataset preparation and general task organization, CG for creating baselines, AN for compiling information on participants methods, AJY for conducting the automatic evaluation, LY, DW, MN, FV, AS, AN, GMDN, RR, PT, MVN, AJY for evaluating the alignment of the test sets, and LY, DW, MN, FV, AS, MO, NM, AN, RB, GMDN, RR, PT, MVN, AJY for conducting the manual evaluation. All authors approved the final version of the manuscript. E-mail for contact: mariana.lara-neves@bfr.bund.de submissions from 15 teams from 6 countries. 1 Introduction Machine translation (MT) is the automatic trans- lation of textual resources from one language to another. It is an important component in many ap- plications and natural language processing (NLP) pipelines in the clinical and biomedical domains. On the one hand, some resources, such as specific biomedical terminologies, are only available for a limited number of languages. English is especially well covered in the Unified Medical Language Sys- tem (UMLS) (Lindberg et al., 1993) while other languages are not (Wilde, 2021). On the other hand, there are many publications written in languages other than English and are therefore inaccessible to researchers who cannot read those languages. This context has been …

Content truncated to 500 words