Representing Verbs with Visual Argument Vectors

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5865–5870 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5865 Representing Verbs with Visual Argument Vectors Irene Sucameli, Alessandro Lenci CoLing Lab, University of Pisa via S. Maria 36 - 56126 Pisa (Italy) irene.sucameli@phd.unipi.it, alessandro.lenci@unipi.it Abstract Is it possible to use images to model verb semantic similarities? Starting from this core question, we developed two textual distributional semantic models and a visual one. We found it particularly interesting and challenging to investigate this Part of Speech since verbs are not often analysed in researches focused on multimodal distributional semantics. After the creation of the visual and textual distributional space, the three models were evaluated in relation to SimLex-999, a gold standard resource. Through this evaluation, we demonstrate that, using visual distributional models, it is possible to extract meaningful information and to effectively capture the semantic similarity between verbs. Keywords: Distributional Semantic, Visual Models, verbs 1. Introduction Imagine that someone offers you something to drink. Even if you do not know what kind of drink it might be, you could make some hypotheses. You could infer that the drink would be contained in a bottle or in a cup and that it would be either water, wine or beer. In the same way, you would consider unrealistic the idea of drinking an apple pie or a piece of chair. This type of educated guess is possible be- cause you are able to derive the semantics of the verb drink from its linguistic and experiential context. For example, water often cooccurs with drink both in linguistic data, such as books, and in our multimodal experience. Distributional Semantics is based on the assumption that words with similar meaning tend to be used within the same linguistic contexts. This hypothesis is implemented by Distributional Semantic Models (DSMs), which repre- sent each lexical element through a distributional vector (Lenci, 2018). The similarity of two words is then calcu- lated based on the position of their vectors within the distri- bution space. Purely linguistic DSMs may be extended to Multimodal DSMs (MDSMs), which combine the informa- tion coded within the linguistic vector with different type of data, the most common of which are visual information extracted from datasets of images (Lazaridou et al., 2015). These models tend to perform better than the ones based only on linguistic data, which show several limitations. In fact, even if linguistic models are able to capture complex linguistic properties, they do not register attributes that for a human being are absolutely intuitive. For example, it is more probable to extract from a text the information that a lemon is sour than that it is yellow, because it is unlikely that a writer would describe such obvious information as lemons are yellow (Baroni and Lenci, 2008; Andrews et al., 2009; Riordan and Jones, 2011). Similarly, if we would like to define an example of obvious information which is not encoded in a verb …

Content truncated to 500 words