Annotating Topic Development in Information Seeking Queries

Annotating Topic Development in Information Seeking Queries Marta Andersson⋆, Adnan ¨Ozt¨urel†, Silvia Pareti† ⋆English Language Department, Stockholm University, Sweden †Google Inc., Brandschenkestrasse 110, 8002 Z¨urich, Switzerland marta.andersson@english.su.se, {ozturel, spareti}@google.com Abstract This paper contributes to the limited body of empirical research in the domain of discourse structure of information seeking queries. We describe the development of an annotation schema for coding topic development in information seeking queries and the initial observations from a pilot sample of query sessions. The main idea that we explore is the relationship between constant and variable discourse entities and their role in tracking changes in the topic progression. We argue that the topicalized entities remain stable across development of the discourse and can be identified by a simple mechanism where anaphora resolution is a precursor. We also claim that a corpus annotated in this framework can be used as training data for dialogue management and computational semantics systems. Keywords: corpus annotation, information structure, information seeking conversational query sessions 1. Introduction Application of NLP techniques on the domain of informa- tion seeking queries is well worth exploring. The conver- sational setting between the query issuer who is seeking information and the dialogue management systems deliver- ing it has a unique discourse structure. Deliverables of re- search on this specific discourse structure can be valuable in improving dialogue management systems. However, there is still very limited research in the literature on this topic mostly because of the lack of available data. This paper contributes to a rather scarce body of empirical data on information-seeking queries (henceforth ISQ). The main goal pursued here is to devise an annotation method- ology that can capture the discourse structure in a set of successive queries where each is information seeking in structure. We believe the methodology we present can serve well for preparing linguistic resources that can be used for training computational semantic applications, such as topic detection systems. The aim of the present study is twofold: (i) to investigate the nature of topic development in discourse in a corpus of information-seeking queries, (ii) to identify the features that crucially participate in topic development in order to describe their role in this process. By “information seeking query sessions” we mean a set of information-seeking queries (the issuer’s input), where co- herent discourse relations between the successive queries can be identified. We argue that in case where no referen- tial ambiguity is present in the context of an information seeking query sessions, the progression of discourse topic can be identified (and also annotated) with a set of sim- ple heuristic rules. However, in the case of referential am- biguity, which may be introduced by anaphora in follow- up queries, disambiguation can be achieved through auto- mated anaphora resolution. Recent advancements in computational semantics deliver methodologies to build wide-coverage systems that can construct meaning representations and carry out robust anaphora resolution (Bos, 2008). We believe our annotation methodology can set the ground for crafting linguistic re- sources that can be used to train specialized …

Content truncated to 500 words