TüKaPo at SemEval-2020 Task 6: Def(n)tly Not BERT: Definition Extraction Using pre-BERT Methods in a post-BERT World

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 724–729 Barcelona, Spain (Online), December 12, 2020. 724 . T¨uKaPo at SemEval-2020 Task 6: Def(n)tly not BERT: Definition Extraction using pre-BERT Methods in a post-BERT World Madeeswaran Kannan∗ Haemanth Shanthi Ponnusamy† Department of Linguistics University of T¨ubingen, Germany Abstract We describe our system (T¨uKaPo) submitted for Task 6: DeftEval, at SemEval 2020. We developed a hybrid approach that combined existing CNN and RNN methods and investigated the impact of purely-syntactic and semantic features on the task of definition extraction, i.e, sentence classification. Our final model achieved a F1-score of 0.6851 in the first subtask. 1 Introduction By all accounts, the first reliable English dictionary was written by Samuel Johnson1 and published on 15 April, 1755. It was 18 inches tall, 20 inches wide when opened, and contained 42,773 entries. It took Dr. Johnson 7 years to complete, and yet it was missing the word contrafibularity, amongst others.2 In the 265 years that have passed since, the world has evolved at a blisteringly fast pace with technology integrating itself ever more closely with our lives. However, the compilation and maintenance of dictionaries and lexicons – one of the most important and authoritative sources of meaning – continues to be the exclusive field of domain experts and lexicographers. Nevertheless, with the recent advances in natural language processing, this area – like many others that deal with human language – has seen a growing interest in automating the development of such resources. Definition extraction is defined as the automatic identification of definitional knowledge in text, modeled as a binary classification problem between definitional and non-definitional text. In the early days of definition extraction, rule-based approaches leveraging linguistic features showed promise. Westerhout (2009) used a combination of linguistic information (n-grams, syntactic features) and structural information (position in sentence, layout) to extract definitions from Dutch texts. Such approaches, however, were found to be dependent on language and domain, and scaled poorly. Later research incorporated machine learning methods to encode lexical and syntactic features as word vectors (Del Gaudio et al., 2014). Noraset et al. (2017) tackled the problem as a language modelling task over learned definition embeddings. Espinosa-Anke et al. (2015) derive feature vectors from entity-linking sources and sense-disambiguated word embeddings. More recently, Anke and Schockaert (2018) use convolutional and recurrent neural networks over syntactic dependencies to achieve very good results on the WCL and W00 datasets (Navigli and Velardi, 2010; Jin et al., 2013). This paper describes our approach of combining existing methods over state-of-the-art techniques that involve the use of contextualized word embeddings such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018) in an attempt to determine if the former still offer avenues of optimization that can help them perform competitively with the latter. ∗ mkannan@sfs.uni-tuebingen.de † haemanth.santhi-ponnusamy@student.uni-tuebingen.de This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. 1https://en.wikipedia.org/wiki/A_Dictionary_of_the_English_Language 2Aardvark was another. 725 1.1 Task Background The DeftEval shared task (Spala et al., …

Content truncated to 500 words