Factuality Assessment as Modal Dependency Parsing

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1540–1550 August 1–6, 2021. ©2021 Association for Computational Linguistics 1540 Factuality Assessment as Modal Dependency Parsing Jiarui Yao1, Haoling Qiu2, Jin Zhao1, Bonan Min2, and Nianwen Xue1 1Brandeis University {jryao, jinzhao, xuen}@brandeis.edu 2Raytheon BBN Technologies {haoling.qiu, bonan.min}@raytheon.com Abstract As the sources of information that we consume everyday rapidly diversify, it is becoming in- creasingly important to develop NLP tools that help to evaluate the credibility of the informa- tion we receive. A critical step towards this goal is to determine the factuality of events in text. In this paper, we frame factuality as- sessment as a modal dependency parsing task that identifies the events and their sources, for- mally known as conceivers, and then deter- mine the level of certainty that the sources are asserting with respect to the events. We crowdsource the first large-scale data set an- notated with modal dependency structures that consists of 353 Covid-19 related news arti- cles, 24,016 events, and 2,938 conceivers.1 We also develop the first modal dependency parser that jointly extracts events, conceivers and constructs the modal dependency structure of a text. We evaluate the joint model against a pipeline model and demonstrate the advan- tage of the joint model in conceiver extraction and modal dependency structure construction when events and conceivers are automatically extracted. We believe the dataset and the mod- els will be a valuable resource for a whole host of NLP applications such as fact checking and rumor detection. 1 Introduction “We’re not just fighting an epidemic; we’re fighting an infodemic.” — Tedros Adhanom, WHO The ongoing COVID-19 pandemic taught us the importance of determining factuality of events at a time when the sources of media we consume have greatly diversified. This is compounded by the fact that the information that we receive from these 1https://github.com/Jryao/modal_ dependency news sources usually does not go through as a rig- orous editing and verification process as traditional media do. The sheer volume of the new media con- tent makes human verification impossible and there is thus an increasing need for NLP tools that help verify statements made in these media sources. To verify if an event has indeed happened we first need to determine the level of certainty with which the event is asserted by the information source, which defaults to the author of a document but can also be another source in the text that the au- thor attributes the information to. The factuality of an event cannot be fully determined without also taking into account the credibility of information source. Consider the text snippet in (1): (1) WBUR: A man in his 20s from Worcester County tested positive Tuesday for the new, apparently more contagious coronavirus vari- ant, public health officials said. The variant was first detected in the United Kingdom, and experts have warned that it could soon be- come widespread in the U.S. Suppose our goal …

Content truncated to 500 words