On the Use of External Data for Spoken Named Entity Recognition

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 724 - 737 July 10-15, 2022 ©2022 Association for Computational Linguistics On the Use of External Data for Spoken Named Entity Recognition Ankita Pasad∗ 2, Felix Wu1, Suwon Shon1, Karen Livescu2, Kyu J. Han1 1ASAPP 2Toyota Technological Institute at Chicago Abstract Spoken language understanding (SLU) tasks involve mapping from speech signals to se- mantic labels. Given the complexity of such tasks, good performance is expected to re- quire large labeled datasets, which are diffi- cult to collect for each new task and domain. However, recent advances in self-supervised speech representations have made it feasible to consider learning SLU models with lim- ited labeled data. In this work, we focus on low-resource spoken named entity recognition (NER) and address the question: Beyond self- supervised pre-training, how can we use exter- nal speech and/or text data that are not anno- tated for the task? We consider self-training, knowledge distillation, and transfer learning for end-to-end (E2E) and pipeline (speech recognition followed by text NER) approaches. We find that several of these approaches im- prove performance in resource-constrained set- tings beyond the benefits from pre-trained rep- resentations. Compared to prior work, we find relative improvements in F1 of up to 16%. While the best baseline model is a pipeline approach, the best performance using external data is ultimately achieved by an E2E model. We provide detailed comparisons and analyses, developing insights on, for example, the ef- fects of leveraging external data on (i) different categories of NER errors and (ii) the switch in performance trends between pipeline and E2E models. 1 Introduction Named entity recognition (NER) is a popular task in natural language processing. It involves detect- ing the named entities and their categories from a text sequence. NER can be used to extract infor- mation from unstructured data, which can also be used as features for other tasks like question an- swering (Chen et al., 2017) and slot filling for task- oriented dialogues (Louvan and Magnini, 2018). ∗Work done during an internship at ASAPP. Baseline + External unlabeled speech + External unlabeled text + External transcribed speech 60 70 80 90 F1 (%) 72.0 68.1 74.977.0 73.171.3 77.679.6 86.0 Pipeline E2E Text NER Figure 1: Improvements in spoken NER with 100 hours of external data of different types. “Pipeline” refers to approaches consisting of speech recognition followed by a text NER model; “E2E” refers to approaches that directly map from speech to NER-tagged text. The “Baseline” and “Text NER” numbers are from previ- ously established baselines (Shon et al., 2022). Thanks to pre-trained text representations such as BERT (Devlin et al., 2019), text-based NER has recently improved greatly (Wang et al., 2021b; Li et al., 2020b). Spoken NER, on the other hand, is not as well-studied. It has the added challenges of continuous-valued and longer input sequences and, at the same time, provides opportunities to take advantage of acoustic cues in the input. …

Content truncated to 500 words