Click or Type: An Analysis of Wizard’s Interaction for Future Wizard Interface Design

Proceedings of the of the EACL 2014 Workshop on Dialogue in Motion (DM), pages 19–27, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics Click or Type: An Analysis of Wizard’s Interaction for Future Wizard Interface Design Srinivasan Janarthanam1, Robin Hill2, Anna Dickinson2, Morgan Fredriksson3 1 School of Mathematical and Computer Sciences, Heriot-Watt University 2 School of Informatics, University of Edinburgh 3 Liquid Media AB, Stockholm sc445@hw.ac.uk Abstract We present an analysis of a Pedestrian Navigation and Information dialogue cor- pus collected using a Wizard-of-Oz inter- face. We analysed how wizards preferred to communicate to users given three differ- ent options: preset buttons that can gen- erate an utterance, sequences of buttons and dropdown lists to construct complex utterances and free text utterances. We present our findings and suggestions for future WoZ design based on our findings. 1 Introduction Wizard-of-Oz environments (WoZ) have been be- come an essential tool for collecting and studying dialogue between humans pretending to be ma- chines and human users in various domains. It is an effective way to collect dialogues between real users and dialogue systems before actually imple- menting the dialogue system. In this framework, participants interact with an expert human oper- ator (known as “Wizard”) who is disguised as a dialogue system. These Wizards replace one or more parts of the dialogue system such as speech recognition, natural language understanding, di- alogue management, natural language generation modules and so on. Real users interact differently with humans and computers. While their expecta- tions with human interlocutors are high and varied, they are ready to adapt and “go easy” on comput- ers during interaction (Pearson et al., 2006). So, in a WoZ framework, the conversation between real users and the Wizards (pretending to be dialogue systems) are of an appropriate type to be used for dialogue system design and not as complex as in human-human conversation. In order to provide a speedy response, most WoZ systems are designed in such a way that re- sponses are hard wired to buttons so that they can be sent to the synthesizer at the touch of a button. However, in order to handle unexpected situations, most WoZ interfaces also have a free text interface that allows the Wizard to type any text to be syn- thesised by the synthesizer. Are free text interfaces used only under unexpected situations? In this pa- per, we analyse how free text interfaces are used by Wizards in a pedestrian tourist navigation and information dialogue and discuss how the results of our analysis be used to inform future WoZ de- signs. These dialogues were collected as a part of SpaceBook EU FP7 project. In Section 2, we present previous work in WoZ interfaces and the domain of pedestrian navigation and information. We then present our WoZ setup and data collection in Section 3 and 4. In Section 5, we present our analysis of the corpus, issues and suggestions in Sections 6 and 7. 2 Related work Wizard-of-Oz (WoZ) frameworks have been used …

Content truncated to 500 words