BeSt: The Belief and Sentiment Corpus

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2460–2467 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2460 BeSt: The Belief and Sentiment Corpus Jennifer Tracey∗, Owen Rambow▷◁, Michael Arrigo∗, Claire Cardie♢, Adam Dalton♡ Hoa Dang♠, Mona Diab⋆, Bonnie Dorr¶, Louise Guthrie℧ Magdalena Markowska▷◁, Smaranda Muresan⊚, Vinodkumar Prabhakaran⟳ Samira Shaikh∢, Tomek Strzalkowski♮, Janyce Wiebe† ∗Linguistic Data Consortium ▷◁Stony Brook University ♢Cornell University ♡IHMC ♠NIST ⋆George Washington University ¶University of Florida ℧University of Texas at El Paso ⊚Columbia University ⟳Stanford University ∢University of North Carolina at Charlotte ♮RPI †Deceased ∗Corresponding author: garjen@ldc.upenn.edu Abstract We present the BeSt corpus, which records cognitive State: who believes what (i.e., factuality), and who has what sentiment towards what. This corpus is inspired by similar source-and-target corpora, specifically MPQA and FactBank. The corpus comprises two genres, newswire and discussion forums, in three languages, Chinese (Mandarin), English, and Spanish. The corpus is distributed through the LDC. 1. Introduction Much of natural language processing (NLP) has been oriented towards extracting propositional content: who did what to whom. Examples include semantic an- notations (e.g., Propbank (Kingsbury et al., 2002) and the related OntoNotes (Hovy et al., 2006) and Abstract Meaning Representation (O’Gorman et al., 2018), and FrameNet (Baker et al., 1998)) and entity-relation- event annotations (such as (Song et al., 2015)). Over the last ten years, interest has grown in not only un- derstanding what content (propositions) is mentioned in text, but propositional attitudes of different agents towards this content (e.g., (Mather et al., 2021)). A propositional attitude is a cognitive attitude, including belief and sentiment, towards a proposition. The set of propositional attitudes of an agent is her cog- nitive state (also called “private state”). The author can use language to report her own propositional attitudes, but she can also use language to report on other agents’ propositional attitudes. One of the goals of NLP is, given a set of propositions mentioned in a text, to also understand who, according to the author, believes in which of these propositions with what level of commit- ment, and who has what sentiment towards the propo- sitions and the entities mentioned in them. The first corpus to address such concerns is the MPQA (Multi-Purpose Question-Answer) corpus of (Wiebe et al., 2005), which grew out of Wiebe’s interest in mod- eling and detecting private state, i.e., cognitive state, which includes sentiment and belief/factuality. Fact- Bank (Saur´ı and Pustejovsky, 2009) simplifies MPQA and concentrates on belief/factuality. The BeSt cor- pus which we present in this paper follows FactBank closely but extends it to sentiment and adds multiple genres and more languages. This paper is structured as follows: Section 2 discusses related work in more detail. In Section 3 and Sec- tion 4 we present the goals and basic conceptual struc- ture of the corpus. Section 5 provides details about the corpus, including annotation mechanism and size, and Section 6 discusses some example annotations. We summarize some initial work using the corpus in Sec- …

Content truncated to 500 words