Language Generation for Multimedia Healthcare Briefings

Language Generation for Multimedia Healthcare Briefings Kathleen R. McKeown Shimei Pan and James Shaw Dept. of Computer Science Columbia University New York, NY 10027, USA kathy, pan, shaw©cs, columbia, edu Abstract This paper identifies issues for language generation that arose in developing a multimedia interface to healthcare data that includes coordinated speech, text and graphics. In order to produce brief speech for time-pressured caregivers, the system both combines related informa- tion into a single sentence and uses ab- breviated references in speech when an unambiguous textual reference is also used. Finally, due to the temporal nature of the speech, the language generation module needs to communicate informa- tion about the ordering and duration of references to other temporal media, such as graphics, in order to allow for coordi- nation between media. 1 Introduction In a hospital setting it can be difficult for care- givers to obtain needed information about patients in a timely fashion. In a Cardiac Intensive Care Unit (ICU), communication regarding patient sta- tus is critical during the hour immediately follow- ing a coronary arterial bypass graft (CABG). It is at this critical point, when care is being trans- ferred from the Operating Room (OR) to the ICU and monitoring is at a minimum, that the pa- tient is most vulnerable to delays in treatment. During this time, there are a number of care- givers who need information about patient status and plans for care, including the ICU nurses who must prepare for patient arrival, the cardiologist who is off-site during the operation, and residents and attendings who will aid in determining post- operative care. The only people who can provide this information are those who were present dur- ing surgery and they are often too busy attending Desmond A. Jordan* Barry A. Allen** Dept. of Anesthesiology* and Medical Informatics Dept.** College of Physicians and Surgeons Columbia University New York, NY 10032 to the patient to communicate much detail. To address this need, we are developing a mul- timedia briefing system, MAGIC (Multimedia Ab- stract Generation for Intensive Care), that takes as input online data collected during the surgical operation as well as information stored in the main databases at Columbia Presbyterian Medical Cen- ter (Roderer and Clayton, 1992). MAGIC gener- ates a multimedia briefing that integrates speech, text, and animated graphics to provide an update on patient status (Dalal et al., 1996a). In this pa- per, we describe the issues that arise for language generation in this context: • Conciseness: The generation process must make coordinated use of speech and text to produce an overview that is short enough for time pressured caregivers to follow, but un- ambiguous in meaning. • Media specific tailoring: Generation must take into account that one output medium is speech, as opposed to the more usual writ- ten language, producing wording and sen- tence structure appropriate for spoken lan- guage. • Coordination with other media: The lan- guage generation process must produce enough information so that speech and text can be coordinated …

Content truncated to 500 words