ACNLP at SemEval-2020 Task 6: A Supervised Approach for Definition Extraction

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 479–486 Barcelona, Spain (Online), December 12, 2020. 479 ACNLP at SemEval-2020 Task 6: A Supervised Approach for Definition Extraction Fabien Caspani Pirashanth Ratnamogan Mathis Linger Mhamed Hajaiej BNP Paribas, France {fabien.caspani, pirashanth.ratnamogan, mathis.linger, mhamed.hajaiej}@bnpparibas.com Abstract We describe our contribution to two of the subtasks of SemEval 2020 Task 6, DeftEval: Extracting term-definition pairs in free text. The system for Subtask 1: Sentence Classification is based on a transformer architecture where we use transfer learning to fine-tune a pretrained model on the downstream task, and the one for Subtask 3: Relation Classification uses a Random Forest classifier with handcrafted dedicated features. Our systems respectively achieve 0.830 and 0.994 F1-scores on the official test set, and we believe that the insights derived from our study are potentially relevant to help advance the research on definition extraction. 1 Introduction SemEval 2020 Task 6 (Spala et al., 2020) is a definition extraction problem divided into three different subtasks, in which the objective is to find the term-definition pairs in free text, possibly spanning across sentences boundaries. In Subtask 1 the system classifies a sentence as containing a definition or not. Subtask 2 consists of labeling each token with their corresponding BIO tags. Finally, in Subtask 3 the system pairs tagged entities, and labels the relation existing between them. The Definition Extraction from Texts (DEFT) corpus (Spala et al., 2019) is the dataset used for this task, and consists of human-annotated data from textbooks on a variety of subjects. Definition extraction methods in the research literature fall into one of the following categories. The earlier research on the subject applies rules-based approaches, such as Klavans and Muresan (2001) whose system extracts definitions from medical publications. Similar approaches can also be found in Cui et al. (2004; 2005) as well as in Westerhout and Monachesi (2007). These methods find definitions by locating words such as is called, means or is, and using grammatical rules on top of that. They generally suffer from low recall, but this issue has been addressed by features-based methods. Fahmi and Bouma (2006) have trained a sentence classifier with features based on bag-of-words and n-grams, position of the sentences in the text and syntactic information. Westerhout (2009) expands the set of features with additional linguistic and structural information, and uses a hybrid approach between a rules-based system and a machine learning classifier. The above approaches do not generalize well to new domains, as rules and handcrafted features are often relevant to specific tasks and time-consuming to create, hence the development of Machine Learning and Deep Learning models. Li et al. (2016) use a Long Short-Term Memory neural (LSTM) network classifier, where features are automatically created from raw input sentences and part-of-speech sequences. Contextual words embeddings depend on the adjacent text, and are pretrained in an unsupervised manner to automatically learn semantic concepts before being used on different downstream tasks (Akbik et al., 2018). Lin and Lu (2018) apply transfer learning to their …

Content truncated to 500 words