A Multimodal Interface for Access to Content in the Home

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 376–383, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics A Multimodal Interface for Access to Content in the Home Michael Johnston AT&T Labs Research, Florham Park, New Jersey, USA johnston@ research. att.com Luis Fernando D’Haro Universidad Politécnica de Madrid, Madrid, Spain lfdharo@die. upm.es Michelle Levine AT&T Labs Research, Florham Park, New Jersey, USA mfl@research. att.com Bernard Renger AT&T Labs Research, Florham Park, New Jersey, USA renger@ research. att.com Abstract In order to effectively access the rapidly increasing range of media content available in the home, new kinds of more natural in- terfaces are needed. In this paper, we ex- plore the application of multimodal inter- face technologies to searching and brows- ing a database of movies. The resulting system allows users to access movies using speech, pen, remote control, and dynamic combinations of these modalities. An ex- perimental evaluation, with more than 40 users, is presented contrasting two variants of the system: one combining speech with traditional remote control input and a sec- ond where the user has a tablet display supporting speech and pen input. 1 Introduction As traditional entertainment channels and the internet converge through the advent of technolo- gies such as broadband access, movies-on-demand, and streaming video, an increasingly large range of content is available to consumers in the home. However, to benefit from this new wealth of con- tent, users need to be able to rapidly and easily find what they are actually interested in, and do so ef- fortlessly while relaxing on the couch in their liv- ing room — a location where they typically do not have easy access to the keyboard, mouse, and close-up screen display typical of desktop web browsing. Current interfaces to cable and satellite televi- sion services typically use direct manipulation of a graphical user interface using a remote control. In order to find content, users generally have to either navigate a complex, pre-defined, and often deeply embedded menu structure or type in titles or other key phrases using an onscreen keyboard or triple tap input on a remote control keypad. These inter- faces are cumbersome and do not scale well as the range of content available increases (Berglund, 2004; Mitchell, 1999). Figure 1 Multimodal interface on tablet In this paper we explore the application of multi- modal interface technologies (See André (2002) for an overview) to the creation of more effective systems used to search and browse for entertain- ment content in the home. A number of previous systems have investigated the addition of unimodal spoken search queries to a graphical electronic program guide (Ibrahim and Johansson, 2002 376 (NokiaTV); Goto et al., 2003; Wittenburg et al., 2006). Wittenburg et al experiment with unre- stricted speech input for electronic program guide search, and use a highlighting mechanism to pro- vide feedback to the user regarding the “relevant” terms the system understood and used to make the query. However, their usability study results show this …

Content truncated to 500 words