Incremental Fine-grained Information Status Classification Using Attention-based LSTMs

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1880–1890, Osaka, Japan, December 11-17 2016. Incremental Fine-grained Information Status Classification Using Attention-based LSTMs Yufang Hou IBM Research Ireland, Dublin, Ireland yhou@ie.ibm.com Abstract Information status plays an important role in discourse processing. According to the hearer’s common sense knowledge and his comprehension of the preceding text, a discourse entity could be old, mediated or new. In this paper, we propose an attention-based LSTM model to address the problem of fine-grained information status classification in an incremental manner. Our approach resembles how human beings process the task, i.e., decide the information status of the current discourse entity based on its preceding context. Experimental results on the ISNotes corpus (Markert et al., 2012) reveal that (1) despite its moderate result, our model with only word embedding features captures the necessary semantic knowledge needed for the task by a large extent; and (2) when incorporating with additional several simple features, our model achieves the competitive results compared to the state-of-the-art approach (Hou et al., 2013) which heavily depends on lots of hand-crafted semantic features. 1 Introduction Information status (IS) (Halliday, 1967; Prince, 1981; Nissim et al., 2004) accounts for the familiarity of a discourse entity according to its accessibility to the hearer at a given point in the text, e.g., old mentions1 are known to the hearer and have been referred to previously; mediated mentions have not been mentioned before but are accessible to the hearer by reference to another old mention or to prior world knowledge; new mentions are “not being recoverable from the preceding discourse” (Halliday, 1967). Information status has attracted a large amount of interests in theoretical linguistics under the frame- work of information structure (Halliday, 1967; Prince, 1981; Prince, 1992; Gundel et al., 1993; Lam- brecht, 1994; Birner and Ward, 1998; Kruijff-Korbayov´a and Steedman, 2003). Many NLP tasks can benefit from knowing information status of discourse entities. Cahill and Riester (2009) improve the performance of generation ranking in German by incorporating features modeling IS. Rahman and Ng (2011) show that a coreference system can profit from IS classification. Baumann and Riester (2013) conduct an empirical study of information status in spoken German and demonstrate that IS can influ- ence prosody in read speech. Hou et al. (2013) model bridging anaphora recognition as a subtask of learning fine-grained information status. In this paper, we focus on classifying IS on written text because many applications which can benefit from IS concentrate on written texts. We follow the IS scheme for written text proposed by Markert et al. (2012). It adopts the three major IS categories (old, new and mediated) from Nissim et al. (2004) and distinguishes six subcategories for mediated. Section 2 provides a brief description of the scheme. We address the task of fine-grained IS classification via an attention-based LSTM model in an incre- mental manner. The model resembles human beings’ cognitive process of determining IS for discourse entities, i.e., during the process of reading a …

Content truncated to 500 words