Listen-Communicate-Show: Spoken Language Command of Agent-based Remote Information Access

Listen-Communicate-Show (LCS): Spoken Language Command of Agent-based Remote Information Access Jody J. Daniels and Benjamin Bell Lockheed Martin Advanced Technology Laboratories 1 Federal Street, A&E 3W Camden, NJ 08102 {jdaniels, bbell@atl.lmco.com} ABSTRACT Listen-Communicate-Show (LCS) is a new paradigm for human interaction with data sources. We integrate a spoken language understanding system with intelligent mobile agents that mediate between users and information sources. We have built and will demonstrate an application of this approach called LCS-Marine. Using LCS-Marine, tactical personnel can converse with their logistics system to place a supply or information request. The request is passed to a mobile, intelligent agent for execution at the appropriate database. Requestors can also instruct the system to notify them when the status of a request changes or when a request is complete. We have demonstrated this capability in several field exercises with the Marines and are currently developing applications of this technology in new domains. Keywords Spoken language understanding, agents, dialogue management. 1. INTRODUCTION An LCS system listens for information requests, communicates both with the user and networked information resources, and shows a tailored visualization to the individual user. The LCS-Marine system employs a spoken language understanding system (SLS) for assisting the user in placing a request and mobile, intelligent agents for information access to implement the LCS paradigm. The SLS converses with the user to generate a request or to check status, amend, or cancel an existing request. Once sufficient information is obtained from the user, the SLS launches an agent to accomplish the requested task. The agent accesses the appropriate databases via whatever network services are available (including existing tactical communications networks). Once the agent's tasks are complete, it returns to the SLS, which generates an appropriate response to the user. The response may be visual, verbal, or a combination, depending on the available devices. 2. SYSTEM OVERVIEW The LCS-Marine system consists of four major components: an SLS, a collection of agents for information access, real-world operational databases, and communications networks to connect the user to the SLS and the agents to the databases. The underlying architecture for the system is the MIT Galaxy II conversational architecture [3]. It is a distributed, component- based middleware product designed to be Òplug and playÓ. Specialized servers handle specific tasks, such as translating audio data to text. All Galaxy II-compliant servers com- municate with each other through a central server known as the Hub. The Hub manages flow control, handles traffic among distributed servers, and provides state maintenance. In the SLS, speech is sent from the Audio I/O server to the Recognizer. The top n recognitions are then parsed, prior context added, and processed using the Natural Language (NL) servers (Frame Construction and Context Tracking) to verify the new input's validity and context. The Turn Manager (TM) determines how to proceed with the conversations and generates a response. NL (Language Generation) converts it to text and the Synthesis server generates the verbal response. The audio server then speaks the waveform file to the user. …

Content truncated to 500 words