Learning the Information Status of Noun Phrases in Spoken Dialogues

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1069–1080, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Learning the Information Status of Noun Phrases in Spoken Dialogues Altaf Rahman and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {altaf,vince}@hlt.utdallas.edu Abstract An entity in a dialogue may be old, new, or mediated/inferrable with respect to the hearer’s beliefs. Knowing the information status of the entities participating in a dia- logue can therefore facilitate its interpreta- tion. We address the under-investigated prob- lem of automatically determining the informa- tion status of discourse entities. Specifically, we extend Nissim’s (2006) machine learning approach to information-status determination with lexical and structured features, and ex- ploit learned knowledge of the information status of each discourse entity for coreference resolution. Experimental results on a set of Switchboard dialogues reveal that (1) incor- porating our proposed features into Nissim’s feature set enables our system to achieve state- of-the-art performance on information-status classification, and (2) the resulting informa- tion can be used to improve the performance of learning-based coreference resolvers. 1 Introduction Information status is not a term unfamiliar to re- searchers working on discourse processing prob- lems. It describes the extent to which a discourse en- tity, which is typically a noun phrase (NP), is avail- able to the hearer given the speaker’s assumptions about the hearer’s beliefs. According to Nissim et al. (2004), a discourse entity can be new, old, or me- diated. Informally, a discourse entity is (1) old to the hearer if it is known to the hearer and has pre- viously been referred to in the dialogue, (2) new if it is unknown to her and has not been previously re- ferred to; and (3) mediated if it is newly mentioned in the dialogue but she can infer its identity from a previously-mentioned entity. Information status is a subject that has received a lot of attention in theoretical linguistics (Halliday, 1976; Prince, 1981; Hajiˇcov´a, 1984; Vallduv´ı, 1992; Steedman, 2000). Knowing the information status of discourse enti- ties can potentially benefit many NLP applications. One such task is anaphora resolution. While there is general belief that definite descriptions are mostly anaphoric, Vieira and Poesio (2000) empirically show that only 30% of these NPs are anaphoric. Without being able to determine whether an NP is anaphoric, an anaphora resolver will attempt to re- solve every NP, potentially damaging its precision. Since new entities are by definition new to the hearer and therefore cannot refer to a previously-introduced NP, knowledge of information status could be used to improve anaphora resolution. Despite the potential usefulness of information status in NLP tasks, there has been little work on learning the information status of discourse entities. To investigate the plausibility of learning informa- tion status, Nissim et al. (2004) annotate a set of Switchboard dialogues with such information1, and subsequently present a rule-based approach and a learning-based approach to acquiring such knowl- edge from the …

Content truncated to 500 words