A Comparative Study of Weighting Schemes for the Interpretation of Spoken Referring Expressions

A Comparative Study of Weighting Schemes for the Interpretation of Spoken Referring Expressions Su Nam Kim, Ingrid Zukerman, Thomas Kleinbauer and Masud Moshtaghi Clayton School of Information Technology, Monash University Clayton, Victoria 3800, Australia firstname.lastname@monash.edu Abstract This paper empirically explores the influ- ence of two types of factors on the in- terpretation of spoken object descriptions: (1) descriptive attributes, e.g., colour and size; and (2) interpretation stages, e.g., syntax and pragmatics. We also investi- gate two schemes for combining attributes when estimating the goodness of an in- terpretation: Multiplicative and Additive. Our results show that the former scheme outperforms the latter, and that the weights assigned to the attributes of a description and the stages of an interpretation influ- ence interpretation accuracy. 1 Introduction Referring expressions have been the topic of con- siderable research in Natural Language Genera- tion (NLG) and psychology. In particular, atten- tion has been paid to the usage of descriptive at- tributes, such as lexical item, colour, size, location and orientation (Section 2). In this paper, we present an empirical study that examines the contribution of two types of fac- tors to the understanding of spoken descriptions: (1) descriptive attributes, such as colour and size; and (2) stages of an interpretation, e.g., syntax and pragmatics. Our study was conducted in the con- text of Scusi?, a Spoken Language Understanding (SLU) system that interprets descriptions of house- hold objects (Zukerman et al., 2008) (Section 3). Given a description such as “the large blue mug”, where the descriptive attributes pertain to colour and size, in the absence of such a mug, should an SLU system prefer a large pink mug or a small blue mug? A preference for the former favours size over colour, while preferring the latter has the opposite effect. Similarly, considering the stages of an interpretation, if an Automatic Speech Rec- ognizer (ASR) produces the text “the played inside the microwave” when a speaker says “the plate in- side the microwave”, should an SLU system pre- fer interpretations comprising objects inside the microwave or interpretations where “played” is considered a verb? A preference for the former favours pragmatics, while a preference for the lat- ter favours the heard text. We represent the contribution of a factor by assigning it a weight — factors with a higher weight are more influential than those with a lower weight; and investigate two methods for learning the weights of the factors pertaining to descriptive attributes and to interpretation stages: steepest as- cent hill climbing and a genetic algorithm (Sec- tion 4). In addition, we consider two schemes for combining descriptive attributes, viz Multiplica- tive and Additive (Section 3.1). Our contribution pertains to the idea of empirically determining the influence of different factors on the interpretation accuracy of an SLU module, the methods for do- ing so, and the analysis of our results. The rest of this paper is organized as follows. Next, we discuss related work. In Section 3, we outline our SLU system and the schemes for com- …

Content truncated to 500 words