STIL - Simultaneous Slot Filling, Translation, Intent Classification, and Language Identification: Initial Results using mBART on MultiATIS++

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 576–581 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 576 STIL - Simultaneous Slot Filling, Translation, Intent Classification, and Language Identification: Initial Results using mBART on MultiATIS++ Jack G. M. FitzGerald Amazon Alexa AI Seattle, WA jgmf@amazon.com Abstract Slot-filling, Translation, Intent classification, and Language identification, or STIL, is a newly-proposed task for multilingual Natural Language Understanding (NLU). By perform- ing simultaneous slot filling and translation into a single output language (English in this case), some portion of downstream system components can be monolingual, reducing de- velopment and maintenance cost. Results are given using the multilingual BART model (Liu et al., 2020) fine-tuned on 7 languages using the MultiATIS++ dataset. When no transla- tion is performed, mBART’s performance is comparable to the current state of the art sys- tem (Cross-Lingual BERT by Xu et al. (2020)) for the languages tested, with better average intent classification accuracy (96.07% versus 95.50%) but worse average slot F1 (89.87% versus 90.81%). When simultaneous transla- tion is performed, average intent classification accuracy degrades by only 1.7% relative and average slot F1 degrades by only 1.2% rela- tive. 1 Introduction Multilingual Natural Language Understanding (NLU), also called cross-lingual NLU, is a tech- nique by which an NLU-based system can scale to multiple languages. A single model is trained on more than one language, and it can accept input from more than one language during inference. In most recent high-performing systems, a model is first pre-trained using unlabeled data for all sup- ported languages and then fine tuned for a specific task using a small set of labeled data (Conneau and Lample, 2019; Pires et al., 2019). Two typical tasks for goal-based systems, such as virtual assistants and chatbots, are intent classifi- cation and slot filling (Gupta et al., 2006). Though intent classification creates a language agnostic output (the intent of the user), slot filling does not. Input 从盐湖城到加州奥克兰的航班 Traditional Output intent: flight slots: (盐湖城, fromloc.cityname), . . . (奥克兰, toloc.cityname), . . . (加州, toloc.statename) STIL Output intent: flight slots: (salt lake city, fromloc.cityname), . . . (oakland, toloc.cityname), . . . (california, toloc.statename) lang: zh Table 1: Today’s slot filling systems do not translate the slot content, as shown in “Traditional Ouput.” With a STIL model, the slot content is translated and lan- guage identification is performed. Instead, a slot-filling model outputs the labels for each of input tokens from the user. Suppose the slot-filling model can handle L languages. Down- stream components must therefore handle all L languages for the full system to be multilingual across L languages. Machine translation could be performed before the slot filling model at system runtime, though the latency would be fully additive, and some amount of information useful to the slot- filling model may be lost. Similarly, translation could occur after the slot-filling model at runtime, but slot alignment between the …

Content truncated to 500 words