End-to-end ASR to jointly predict transcriptions and linguistic annotations
21725Authors
nan
Abstract
AbstractWe propose a Transformer-based sequence-to-sequence model for automatic speech recognition (ASR) capable of simultaneously transcribing and annotating audio with linguistic information such as phonemic transcripts or part-of-speech (POS) tags. Since linguistic information is important in natural language processing (NLP), the proposed ASR is especially useful for speech interface applications, including spoken dialogue systems and speech translation, which combine ASR and NLP. To produce linguistic annotations, we train the ASR system using modified training targets: each grapheme or multi-grapheme unit in the target transcript is followed by an aligned phoneme sequence and/or POS tag. Since our method has access to the underlying audio data, we can estimate linguistic annotations more accurately than pipeline approaches in which NLP-based methods are applied to a hypothesized ASR transcript. Experimental results on Japanese and English datasets show that the proposed ASR system is capable of simultaneously producing high-quality transcriptions and linguistic annotations.
Keywords
Full Content
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 151–161 June 6–11, 2021. ©2021 Association for Computational Linguistics 151 Backtranslation Feedback Improves User Confidence in MT, Not Quality Vilém Zouhar1, Michal Novák1, Matúš Žilinec1, Ondˇrej Bojar1, Mateo Obregón2, Robin L. Hill2, Frédéric Blain3,4, Marina Fomicheva3, Lucia Specia3, Lisa Yankovskaya5 1Charles University, Czech Republic, 2University of Edinburgh, 3University of Sheffield, 4University of Wolverhampton, United Kingdom, 5University of Tartu, Estonia {zouhar,mnovak,zilinec,bojar}@ufal.cuni.cz {mateo.obregon,r.l.hill}@ed.ac.uk f.blain@wlv.ac.uk {m.fomicheva,l.specia}@sheffield.ac.uk lisa.yankovskaya@ut.ee Abstract Translating text into a language unknown to the text’s author, dubbed outbound translation, is a modern need for which the user experi- ence has significant room for improvement, beyond the basic machine translation facility. We demonstrate this by showing three ways in which user confidence in the outbound trans- lation, as well as its overall final quality, can be affected: backward translation, quality esti- mation (with alignment) and source paraphras- ing. In this paper, we describe an experiment on outbound translation from English to Czech and Estonian. We examine the effects of each proposed feedback module and further focus on how the quality of machine translation sys- tems influence these findings and the user per- ception of success. We show that backward translation feedback has a mixed effect on the whole process: it increases user confidence in the produced translation, but not the objective quality. 1 Introduction When dealing with machine translation (MT) on the web, most of the attention of the research com- munity is paid to inbound translation. In this sce- nario, the recipients are aware of the MT process, and thus it is their responsibility to interpret and understand the translated content correctly. For an MT system, it is sufficient to achieve such quality that allows a recipient to get the gist of the mean- ing of texts on webpages. For outbound translation, it is the other way round: the responsibility to create the content in the way that it is correctly interpreted by a recip- ient lies on the authors of the message. The main issue is that the target language might be entirely unknown to them. Prototypically it is communica- tion by email, filling in foreign language forms, or involving some other kind of interactive medium. The focus in this scenario is placed not only on producing high-quality translations but also on re- assuring the author that the MT output is correct. One of the approaches to improving both qual- ity and authors’ confidence, first employed in this scenario by Zouhar and Bojar (2020), is to provide cues that indicate the quality of MT output as well as suggest possible rephrasing of the source. They may include backward translation to the source language, highlighting of the potentially problem- atic parts of the input, or suggesting paraphrases. Except for preliminary work by Zouhar and Novák (2020), the impact of individual cues has not yet been properly explored. In this paper, we present the results of a new experiment on outbound translation. …
Content truncated to 500 words
Labeled Text
AbstractWe propose a Transformer-based sequence-to-sequence model for automatic speech recognition (ASR) capable of simultaneously transcribing and annotating audio with linguistic information such as phonemic transcripts or part-of-speech (POS) tags. Since linguistic information is important in natural language processing (NLP), the proposed ASR is especially useful for speech interface applications, including spoken dialogue systems and speech translation, which combine ASR and NLP. To produce linguistic annotations, we train the ASR system using modified training targets: each grapheme or multi-grapheme unit in the target transcript is followed by an aligned phoneme sequence and/or POS tag. Since our method has access to the underlying audio data, we can estimate linguistic annotations more accurately than pipeline approaches in which NLP-based methods are applied to a hypothesized ASR transcript. Experimental results on Japanese and English datasets show that the proposed ASR system is capable of simultaneously producing high-quality transcriptions and linguistic annotations.
Publication Details
- Paper ID:
- 21725
- Year:
- 2021
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan