Fine-grained Information Status Classification Using Discourse Context-Aware BERT

Proceedings of the 28th International Conference on Computational Linguistics, pages 6101–6112 Barcelona, Spain (Online), December 8-13, 2020 6101 Fine-grained Information Status Classification Using Discourse Context-Aware BERT Yufang Hou IBM Research Europe, Ireland yhou@ie.ibm.com Abstract Previous work on bridging anaphora recognition (Hou et al., 2013a) casts the problem as a subtask of learning fine-grained information status (IS). However, these systems heavily depend on many hand-crafted linguistic features. In this paper, we propose a simple discourse context-aware BERT model for fine-grained IS classification. On the ISNotes corpus (Markert et al., 2012), our model achieves new state-of-the-art performance on fine-grained IS classification, obtaining a 4.8 absolute overall accuracy improvement compared to Hou et al. (2013a). More importantly, we also show an improvement of 10.5 F1 points for bridging anaphora recognition without using any complex hand-crafted semantic features designed for capturing the bridging phenomenon. We further analyze the trained model and find that the most attended signals for each IS category correspond well to linguistic notions of information status. 1 Introduction Information Structure (Halliday, 1967; Prince, 1981; Prince, 1992; Gundel et al., 1993; Lambrecht, 1994; Birner and Ward, 1998; Kruijff-Korbayov´a and Steedman, 2003) studies structural and semantic properties of a sentence according to its relation to the discourse context. Information structure affects how discourse entities are referred to in a text, which is known as Information Status (Halliday, 1967; Prince, 1981; Nissim et al., 2004). Specifically, information status (IS henceforth) reflects the accessibility of a discourse entity based on the evolving discourse context and the speaker’s assumption about the hearer’s knowledge and beliefs. For instance, according to Markert et al. (2012), old mentions1 refer to entities that have been referred to previously; mediated mentions have not been mentioned before but are accessible to the hearer by reference to another old mention or to prior world knowledge; and new mentions refer to entities that are introduced to the discourse for the first time and are not known to the hearer before. In this paper, we mainly follow the IS scheme proposed by Markert et al. (2012) and focus on learning fine-grained IS on written texts. A mention’s semantic and syntactic properties can signal its information status. For instance, indefinite NPs tend to be new and pronouns are likely to be old. Moreover, referential patterns of how a mention is referred to in a sentence also affect this mention’s IS. In Example 1, “Friends” is a bridging anaphor even if we do not know the antecedent (i.e., she); while the information status for “Friends” in Example 2 is mediated/worldKnowledge. Section 3.1 analyzes the characteristics of each IS category and the relations between IS and discourse context. (1) [She]antecedent made money, but spent more. Friends pitched in. (2) Friends are part of the glue that holds life and faith together. In this work, we propose a simple yet effective discourse context-aware self-attention model based on BERT (Devlin et al., 2019) for fine-grained IS classification. We find that the sentence containing the target mention as well as the …

Content truncated to 500 words