Comparing two different bidirectional versions of the limited-domain medical spoken language translator MedSLT

Comparing two different bidirectional versions of the limited-domain medical spoken language translator MedSLT Marianne Starlander1, Pierrette Bouillon1, Glenn Flores2, Manny Rayner1, Nikos Tsourakis1 (1) University of Geneva, TIM/ISSCO/ETI {Pierrette.Bouillon, Emmanuel.Rayner, Nikolaos.Tsourakis}@issco.unige.ch, Marianne.Starlander@eti.unige.ch (2) UT Southwestern Medical Center, Children’s Medical Center of Dallas Glenn.Flores@UTsouthwestern.edu Abstract. This paper reports preliminary results of an evaluation during which two different bidirectional versions of the limited-domain medical spoken language translator MedSLT were compared in a hospital setting. The more restricted version (V.1) only allows Yes-No answers and short elliptical sentences, while the less restricted version (V.2) allows Yes-No answers, short elliptical sentences and full sentences. Although WER is marginally better for V.1, task performance is marginally worse. There appear to be two main reasons for this disparity; short sentences are often badly recognised and patients tend to find it difficult to limit themselves to ellipsis, even if they receive clear instructions about not using full sentences. 1. Introduction Today, an ideal speech-to-speech translation system enabling a healthcare provider to communicate naturally with a patient across two different languages during a medical encounter is still well beyond what is possible given the state of the art. It is thus necessary to find good tradeoffs, and develop methodologies to build and objectively compare different currently realisable architectures. In this paper, we compare two different versions of the bidirectional MedSLT system, a medical speech to speech translator, which differ in terms of grammatical coverage. In the first version (V.1), the patient (Pat) can only answer the physician’s (Phy) question using elliptical answers (Example 1). The second (V.2), less restricted version, enables the patient to answer with either elliptical answers or full sentences (Example 2). (1) Phy: Where is the pain? Tran: ¿dónde le duele? Pat: en la garganta Trans: I have a sore throat Phy: For how long have you had a sore throat? Trans: ¿desde cuándo le duele la garganta? Pat: Desde hace más de una semana Trans: I have had a sore throat for more than one week Example 1. Dialogue sample with Version 1 (2) Phy: Where is the pain? Tran: ¿dónde le duele? Pat: Me duele la garganta Trans: I have a sore throat Phy: How long have you had a sore throat? Trans: ¿desde cuándo le duele la garganta? 12th EAMT conference, 22-23 September 2008, Hamburg, Germany 176 Pat: Me duele desde hace dos días Trans: I have experienced the pain for two days Example 2. Dialogue sample with Version 2 The aim of this evaluation is to determine the impact of the restriction in coverage, both in terms of objective system metrics (WER) and also user-oriented performance metrics (task completion time). In the rest of the paper, we first give a quick overview of the MedSLT system (Section 2). We then explain how the two versions of the system can be built using the Regulus platform (Section 3). Finally, we provide some preliminary results from an evaluation comparing the two versions of the system, which was undertaken at Children’s Medical Center Dallas …

Content truncated to 500 words