Extracting Possessions from Social Media: Images Complement Language

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 663–672, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 663 Extracting Possessions from Social Media: Images Complement Language Dhivya Chinnappa, Srikala Murugan and Eduardo Blanco Human Intelligence and Language Technologies Lab University of North Texas Denton, Texas 76203 {dhivyainfantchinnappa,srikalamurugan}@my.unt.edu, eduardo.blanco@unt.edu Abstract This paper describes a new dataset and exper- iments to determine whether authors of tweets possess the objects they tweet about. We work with 5,000 tweets and show that both humans and neural networks benefit from images in addition to text. We also introduce a sim- ple yet effective strategy to incorporate visual information into any neural network beyond weights from pretrained networks. Specifi- cally, we consider the tags identified in an im- age as an additional textual input, and leverage pretrained word embeddings as usually done with regular text. Experimental results show this novel strategy is beneficial. 1 Introduction Social media are platforms for sharing informa- tion online. Social media posts and online be- havior in general (e.g., Facebook likes, following other users in Twitter) have been shown to predict human traits (Burke et al., 2010; Schwartz et al., 2013). Many social media posts include an image alongside text, and the percentage keeps growing as doing so boosts user engagement (Patel, 2016). Pictures and text in social media usually com- plement each other. Thus, even if the information of interest can be understood from either commu- nication modality, considering both is beneficial. Consider the tweet in Figure 1. The text indi- cates that Arnold (the author of the tweet) goes on bike rides when he travels. The image shows him riding a bike, indicating that he was riding a bike when he tweeted thus he was in posses- sion of a bike. On the other hand, if the picture were a screenshot of his Twitter posting statistics, Arnold most likely would not be in possession of a bike when tweeting, but rather sharing a log of his previous trips with his followers. In this paper, we extract possession relations from social media Figure 1: Sample tweet with text and an image. We can infer that Arnold possessed a bike when he tweeted. posts containing both text and images. Posses- sion is an asymmetric semantic relation between two entities, where one entity (the possessee) be- longs to the other entity (the possessor) (Stassen, 2009). Following the literature, we consider not only ownership, but also control possessions. In control possessions, the possessor has temporary control of the possessee but not necessarily own- ership (Tham, 2004), e.g., Bill borrowed the ozone generator from John. While we do not explore any, extracting pos- sessions has many potential applications. For ex- ample, possessions could help to reveal hobbies and to find people with similar interests. Posses- sions could also improve recommender systems. For example, people without cars are unlikely to be interested in oil changes and auto …

Content truncated to 500 words