Speech Translation Systems as a Solution for a Wireless Earpiece

Speech Translation Systems as a Solution for a Wireless Earpiece Nicholas Ruiz, Andrew Ochoa, Jainam Shah, William Goethels, Sergio DelRio Diaz Waverly Labs, Brooklyn, NY, USA {nick,andrew,jainam,william,sergio}@waverlylabs.com Abstract The advances of deep learning approaches in automatic speech recognition (ASR) and machine translation (MT) have al- lowed for levels of accuracy that move speech translation closer to being a com- mercially viable alternative interpretation solution. In addition, recent improve- ments in micro-electronic mechanical sys- tems, microphone arrays, speech process- ing software, and wireless technology have enabled speech recognition software to capture higher quality speech input from wireless earpiece products. With this in mind, we introduce and present a wear- able speech translation tool called Pilot, which uses these systems to translate lan- guage spoken within the proximity of a user wearing the wireless earpiece. 1 What is Pilot? The Pilot Translating Earpiece is a sophisticated earbud which uses dual microphones and cus- tom noise cancelling algorithms to produce clear speech before it is passed through our mobile app and to our speech translation engine in the cloud. It relays speech translation very quickly with min- imal latency. Pilot consists of two translation ear- buds that pair with custom speech translation soft- ware for Android or iOS. Pilot allows consumers to share their secondary earbud with a conversa- tion partner for face-to-face simultaneous speech translation and currently supports 15 languages1. c⃝ 2018 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. 1Arabic, Chinese (Mandarin, Cantonese), English, French, German, Hindi, Italian, Japanese, Korean, Portuguese, Rus- 2 How Does Pilot Work? Pilot operates in two modes: Converse and Lis- ten. As the primary use case, Converse mode al- lows multi-party conversations with transcriptions logged in the app. In a one-on-one conversation, users can share their secondary earbud with a part- ner and quickly pair it with the partner’s phone. Currently in beta, Listen mode adapts the micro- phone firmware settings to pick up ambient sound and performs far-field ASR and MT. Pilot uses sev- eral speech translation paradigms, depending on the language pair, either by running ASR and MT sequentially, or as tightly coupled speech transla- tion2. Translations are primarily run on the server, while the app is responsible for routing the audio to and from the earpiece. Practical challenges Bluetooth: Convention- ally, Android and iOS devices are limited to one microphone connection at a time. Although rout- ing the partner’s earbud recordings through the same phone is possible, it requires low-level kernel programming to implement. Our team will resolve this issue in a future release. Microphone pick-up: Occasionally a conversation partner’s speech can be picked up by the user’s earbud, and vice-versa. While digital signal processing can eliminate some of this effect, the position, distance, and power of the speech must be taken into account. As the provider of one of the first translation wearables to market, we are eager to how learn translation technology affects situational dialogue without an interpreter present. While our …

Content truncated to 500 words