Neural Networks approaches focused on French Spoken Language Understanding: application to the MEDIA Evaluation Task

Proceedings of the 28th International Conference on Computational Linguistics, pages 2722–2727 Barcelona, Spain (Online), December 8-13, 2020 2722 Neural Networks approaches focused on French Spoken Language Understanding: application to the MEDIA Evaluation Task Sahar Ghannay† Christophe Servan‡ Sophie Rosset† †Universit´e Paris-Saclay, CNRS, LIMSI, 91405 Orsay, France lastname@limsi.fr ‡ QWANT 61 rue de Villier 92200 Neuilly-sur-Seine, France inital.lastname@qwant.com Abstract In this paper, we present a study on a French Spoken Language Understanding (SLU) task: the MEDIA task. Many works and studies have been proposed for many tasks, but most of them are focused on English language and tasks. The exploration of a richer language like French within the framework of a SLU task implies to recent approaches to handle this difficulty. Since the MEDIA task seems to be one of the most difficult, according to several previous studies, we propose to explore Neural Networks approaches focusing of three aspects: firstly, the Neu- ral Network inputs and more specifically the word embeddings; secondly, we compared French version of BERT against the best setup through different ways; Finally, the comparison against State-of-the-Art approaches. Results show that the word embeddings trained on a small cor- pus need to be updated during SLU model training. Furthermore, the French BERT fine-tuned approaches outperform the classical Neural Network Architectures and achieves state of the art results. However, the contextual embeddings extracted from one of the French BERT approaches achieve comparable results in comparison to word embedding, when integrated into the proposed neural architecture. 1 Introduction Spoken language understanding (SLU) module is a key component for a spoken language dialogue sys- tem. It consists on semantically analyse user queries and identifies text spans that mention semantic concepts. SLU task can fall into three sub-tasks: domain classification, intent classification, and slot- filling (Tur and Mori, 2011). The latter is the task that interests us in this study. Over the past five years, the studies developed for SLU task are based on neural network architec- tures (Yao et al., 2014; Mesnil et al., 2015; Guo et al., 2014; Zhang and Wang, 2016; Dinarelli et al., 2017; Simonnet et al., 2017; Korpusik et al., 2019; Ghannay et al., 2020). Recent approaches take ben- efit from contextual or language model embeddings such as BERT (Devlin et al., 2019). Korpusik et al. (2019) investigated the transfer ability of a pre-trained BERT representation for English SLU tasks. But, as far as we know, there are no such studies on a French SLU task. Following Ghannay et al. (2020)’s study, many avenues can be explored. In their study, the word embeddings have been frozen during training (are not updated), since Lebret et al. (2013) show that fine- tuned word embeddings show very similar performance and provide comparable results. However, the evaluation of whether updating the embeddings during SLU model training improves or not the results, for SLU task, is less studied. In addition, their SLU model is fed only with word embeddings, and they did not use any additional features, thus there are some rows …

Content truncated to 500 words