Analysing the Impact of Supervised Machine Learning on Automatic Term Extraction: HAMLET vs TermoStat

Proceedings of Recent Advances in Natural Language Processing, pages 1012–1021, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_117 1012 Analysing the Impact of Supervised Machine Learning on Automatic Term Extraction: HAMLET vs TermoStat Ayla Rigouts Terryn*, Patrick Drouin**, Veronique Hoste* and Els Lefever* *LT3 Language and Translation Technology Team, Ghent University Groot-Brittanni¨elaan 45, 9000 Gent; name.surname@ugent.be **Observatoire de Linguistique Sens-Texte, Universit´e de Montr´eal Succ. Centre-ville, Montr´eal, QC, H3C 3J7; patrick.drouin@umontreal.ca Abstract Traditional approaches to automatic term extraction do not rely on machine learn- ing (ML) and select the top n ranked can- didate terms or candidate terms above a certain predefined cut-off point, based on a limited number of linguistic and statis- tical clues. However, supervised ML ap- proaches are gaining interest. Relatively little is known about the impact of these supervised methodologies; evaluations are often limited to precision, and sometimes recall and f1-scores, without information about the nature of the extracted candi- date terms. Therefore, the current paper presents a detailed and elaborate analysis and comparison of a traditional, state-of- the-art system (TermoStat) and a new, su- pervised ML approach (HAMLET), using the results obtained for the same, manually annotated, Dutch corpus about dressage. 1 Introduction Automatic term extraction (ATE), also known as automatic term recognition (ATR), has long been an established task within the field of natural lan- guage processing. It can be used both in its own right, to automatically obtain a list of candidate terms (cts) from a specialised corpus, or as a pre- processing step for other tasks, such as machine translation (Wolf et al., 2011). The traditional method for ATE is a hybrid approach, combining both linguistic and statistical information. In a first step, linguistic preprocessing is performed and a preliminary list of cts is produced based on part- of-speech (POS) patterns. Next, statistical met- rics are applied to measure termhood (to what de- gree a term is related to the domain) and unithood for multi-word terms (whether the individual to- kens combine to form a lexical unit) (Kageura and Umino, 1996). These metrics are used to sort the cts based on their likelihood to be actual terms. To filter the list, one can either determine a cut-off value or select the top n or top n percent of terms. As a final step, manual validation is required. This has been a standard methodology for some time (Daille, 1994) and is still used by state-of- the-art systems such as TermoStat (Drouin, 2003) and TExSIS (Macken et al., 2013). However, the problem with these methodologies is determining the cut-off point (Lopes and Vieira, 2015) and combining multiple features (e.g., separate mea- sures for termhood and unithood). It has become clear that multiple evidence (i.e. combining multi- ple features) is highly beneficial for ATE (Dobrov and Loukachevitch, 2011; Loukachevitch, 2012). Supervised machine learning (ML) methodologies are now being used in answer to these problems. By automatically learning an optimal combination of features and cut-off points, many features can be efficiently combined. One of the biggest hurdles for the …

Content truncated to 500 words