TWEET-FID: An Annotated Dataset for Multiple Foodborne Illness Detection Tasks

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6212–6222 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6212 TWEET-FID: An Annotated Dataset for Multiple Foodborne Illness Detection Tasks Ruofan Hu1, Dongyu Zhang1, Dandan Tao2, Thomas Hartvigsen3, Hao Feng4, Elke Rundensteiner1 1Data Science Program, Worcester Polytechnic Institute, Worcester, MA, USA 2Vanke School of Public Health, Tsinghua University, Beijing, China 3CSAIL, Massachusetts Insititute of Technology, Cambridge, MA, USA 4Dept. of Food Science & Human Nutrition, Univ. of Illinois, Urbana-Champaign, Urbana, IL, USA {rhu, dzhang5, rundenst}@wpi.edu, tomh@mit.edu, {dtao2, haofeng}@illinois.edu Abstract Foodborne illness is a serious but preventable public health problem – with delays in detecting the associated outbreaks resulting in productivity loss, expensive recalls, public safety hazards, and even loss of life. While social media is a promising source for identifying unreported foodborne illnesses, there is a dearth of labeled datasets for developing effective outbreak detection models. To accelerate the development of machine learning-based models for foodborne outbreak detection, we thus present TWEET-FID (TWEET-Foodborne Illness Detection), the first publicly available annotated dataset for multiple foodborne illness incident detection tasks. TWEET-FID collected from Twitter is annotated with three facets: tweet class, entity type, and slot type, with labels produced by experts as well as by crowdsource workers. We introduce several domain tasks leveraging these three facets: text relevance classification (TRC), entity mention detection (EMD), and slot filling (SF). We describe the end-to-end methodology for dataset design, creation, and labeling for supporting model development for these tasks. A comprehensive set of results for these tasks leveraging state-of-the-art single- and multi-task deep learning methods on the TWEET-FID dataset are provided. This dataset opens opportunities for future research in foodborne outbreak detection. Keywords: Foodborne Illness Detecion, Dataset, Social Media, Crowdsourcing, Multi-task Learning 1. Introduction Foodborne illnesses continue to threaten public health. Approximately 1 in 6 Americans (or 48 million people) are sickened by foodborne illness each year (Scallan et al., 2011). Foodborne illnesses lead to loss of pro- ductivity, medical expenses, and even death. The an- nual economic cost caused by foodborne illnesses in the United States is estimated to be between $14 to $60 billion (Hoffmann and Scallan Walter, 2020; Scharff, 2018). Detecting foodborne illnesses early promises to reduce the risk and curtail the outbreak. In the past decade, it has been recognized that user-generated public posts on social media or review platforms can play an im- portant role in the surveillance of unreported cases of foodborne illness; Surveillance applications are being piloted by local health agencies, including mining data from Twitter in New York City (Harrison et al., 2014), Las Vegas (Sadilek et al., 2016), and consumer review sites such as Yelp in San Francisco (Schomberg et al., 2016) and New York City (Effland et al., 2018). How- ever, these applications tend to focus only on a ’post- level’ inspection, where a followed-up labor-intensive manual phase is deployed to extract detailed informa- tion from potential posts related to foodborne illness af- ter they were selected …

Content truncated to 500 words