Understanding Feature Focus in Multitask Settings for Lexico-semantic Relation Identification

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2762–2772 August 1–6, 2021. ©2021 Association for Computational Linguistics 2762 Understanding Feature Focus in Multitask Settings for Lexico-semantic Relation Identification Houssam Akhmouch Normandie Univ, UNICAEN ENSICAEN, CNRS, GREYC Cr´edit Agricole Brie Picardie France Ga¨el Dias Normandie Univ, UNICAEN ENSICAEN, CNRS, GREYC France Jose G. Moreno Universit´e de Toulouse IRIT UMR 5505 CNRS France Abstract Discovering whether words are semantically related and identifying the specific semantic re- lation that holds between them is of crucial im- portance for automatic reasoning on text data. For that purpose, different methodologies have been proposed that either (1) tackle feature en- gineering, (2) fine-tune latent semantic spaces, or (3) take advantage of cognitive links be- tween semantic relations in multitask settings. In this paper, we investigate how feature en- gineering and multitask architectures can be improved and consequently combined to iden- tify lexico-semantic relations. Evaluation re- sults over a set of gold-standard datasets show that (1) combinations of similar features are beneficial (feature sets), (2) asymmetric distri- butional features are a strong cue to discrimi- nate asymmetric relations as well as they play an important role in multitask architectures, (3) shared-private models improve over binary and fully-shared classifiers as well as they cor- rectly balance the focus on features between private and shared layers1. 1 Introduction The ability to automatically identify lexico- semantic relations is an important issue for In- formation Retrieval and Natural Language Pro- cessing applications such as question answering (Dong et al., 2017), query expansion (Kathuria et al., 2017), or text summarization (Gambhir and Gupta, 2017). Lexico-semantic relations embody symmetric and asymmetric linguistic phenomena such as synonymy (e.g. phone ↔ telephone), co- hyponymy (e.g. phone ↔ monitor), hypernymy (e.g. phone → speakerphone) or meronymy (e.g. 1Both the code and the datasets are avail- able at https://github.com/Houssam93/ Feature-Focus-in-Multi-Task-Learning-NLP for reproducibility. phone → mouthpiece), but more can be enumer- ated (Vylomova et al., 2016). Most approaches focus on modeling a single se- mantic relation and consist in deciding whether a given relation r holds between a pair of words (w1, w2). The vast majority of efforts (Shwartz et al., 2016; Vuli´c and Mrkˇsi´c, 2018; Wang and He, 2020) concentrate on hypernymy which is the key organization principle of semantic memory, but studies exist on antonymy (Nguyen et al., 2017b; Ali et al., 2019), meronymy (Glavaˇs and Ponzetto, 2017) and co-hyponymy (Jana et al., 2020). Within this scope, different strategies have been proposed that either define new features (Santus et al., 2017; Vu and Shwartz, 2018) or build specific latent se- mantic spaces (Nguyen et al., 2017a; Rei et al., 2018; Wang and He, 2020) for the relation at hand. More recently, multitask strategies have been proposed, which consist in concurrently learning correlated lexico-semantic relations (Attia et al., 2016; Balikas et al., 2019; Bannour et al., 2020), the underlying idea being that if two (or more) tasks are cognitively interlinked, a learning architecture should improve its generalization ability by tak- ing into account the shared …

Content truncated to 500 words