HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 125–129 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 125 HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings Saba Anwar⋆, Dmitry Ustalov†, Nikolay Arefyev‡, §, Simone Paolo Ponzetto†, Chris Biemann⋆, and Alexander Panchenko⋆,⋄ ⋆Language Technology Group, Department of Informatics, University of Hamburg, Germany ⋄Skolkovo Institute of Science and Technology, Russia {anwar,biemann,panchenko}@informatik.uni-hamburg.de †Data and Web Science Group, University of Mannheim, Germany {dmitry,simone}@informatik.uni-mannheim.de ‡Samsung R&D Institute Russia §Lomonosov Moscow State University, Russia narefyev@cs.msu.ru Abstract We present our system for semantic frame in- duction that showed the best performance in Subtask B.1 and finished as the runner-up in Subtask A of the SemEval 2019 Task 2 on un- supervised semantic frame induction (Qasem- iZadeh et al., 2019). Our approach separates this task into two independent steps: verb clus- tering using word and their context embed- dings and role labeling by combining these embeddings with syntactical features. A sim- ple combination of these steps shows very competitive results and can be extended to pro- cess other datasets and languages. 1 Introduction Recent years have seen a lot of interest in computational models of frame semantics, with the availability of annotated sources like Prop- Bank (Palmer et al., 2005) and FrameNet (Baker et al., 1998). Unfortunately, such annotated re- sources are very scarce due to their language and domain specificity. Consequently, there has been work that investigated methods for unsupervised frame acquisition and parsing (Lang and Lapata, 2010; Modi et al., 2012; Kallmeyer et al., 2018; Ustalov et al., 2018). Researchers have used different approaches to induce frames, includ- ing clustering verb-specific arguments as per their roles (Lang and Lapata, 2010), subject-verb-object triples (Ustalov et al., 2018), syntactic depen- dency representation using dependency formats like CoNLL (Modi et al., 2012; Titov and Kle- mentiev, 2012), and latent-variable PCFG mod- els (Kallmeyer et al., 2018). The SemEval 2019 task of semantic frame and role induction consists of three subtasks: (A) learning the frame type of the highlighted verb from the context in which it has been used; (B.1) clustering the highlighted arguments of the verb into specific roles as per the frame type of that verb, e.g., Buyer, Goods, etc.; (B.2) clustering the arguments into generic roles as per VerbNet classes (Schuler, 2005), without considering the frame type of the verb, i.e., Agent, Theme, etc. Our approach to frame induction is similar to the word sense induction approach by Arefyev et al. (2018), which uses tf–idf-weighted context word embeddings for a shared task on word sense induction by Panchenko et al. (2018). In this unsu- pervised task, our approach for clustering mainly consists of exploring the effectiveness of already available pre-trained models.1 Main contributions of this paper are: 1. a method that uses contextualized distribu- tional word representations (embeddings) for grouping verbs to frame type clusters (Sub- task A); 2. a method that combines word and con- text embeddings for clustering arguments of verbs to frame slots (Subtasks …

Content truncated to 500 words