An End-to-End Multilingual System for Automatic Minuting of Multi-Party Dialogues

An End-to-End Multilingual System for Automatic Minuting of Multi-Party Dialogues Aakash Bhatnagar&, Nidhir Bhavsar$, Muskaan Singh# and Petr Motlicek# & Boston University, Boston, Massachusetts $University of Potsdam, Potsdam, Germany # Speech and Audio Processing Group, IDIAP Research Institute, Martigny, Switzerland aakash07@bu.edu, bhavsar@uni-potsdam.de, (msingh,petr.motlicek)@idiap.ch Abstract In this paper, we present a pipeline for auto- matic minuting. This pipeline is an end-to-end system for minuting the multiparty dialogues of meetings. It provides multilingual com- munication and collaboration, with a specific focus on Natural Language Processing (NLP) technologies: Automatic Speech Recognition (ASR), Machine Translation (MT), Automatic Minuting (AM), Topic Modelling (TM), and Named Entity Recognition (NER). Our summarization model achieves a ROUGE-1 score of 0.45, a BLEU score of 7.069, and a BERT score of 0.673. Our translation model also achieves a high average BERT score of 0.848 across five different languages (de,fr, en, it, and hi). We make our code available at https://github.com/aakash0017/ Paclic-summarization-pipeline 1 Introduction Since the COVID-19 epidemic, a sizeable portion of the working population—particularly those em- ployed in the information technology (IT) sector and academia—has expanded dramatically in vir- tual meetings. Meetings are, without a doubt, the most important element in fostering teamwork and effective back-and-forth communication. There are numerous Natural Language Processing (NLP) technologies available that give users a complete online interaction experience. The interpretation of these online interactions during remote conferences or meetings is crucial in the globally interconnected world of today. Summarizing meetings in the form of structured minutes from speech and it can potentially save up to 80% of time. We realized that generating meeting minutes is a task that is still performed manually and requires a lot of time. Through this paper, we try to solve three major problems encountered during multi- party dialogues via our proposed system. First, as mentioned above, manually generating minutes consumes much time. Each annotator has to go through hours of recording before writing minutes. Also, each annotator may have a different vocabulary and style, leading to inconsistency in the meeting-minutes format. We try to solve this problem by generating meeting minutes in a consis- tent format for multi-party conversation(MPC). We incorporate large pre-trained transformer models fine-tuned on MPC meeting datasets. Second, as globalization is increasing, compa- nies have offices worldwide. Hence, to overcome the language barrier, there is a need for a system that can provide translation of meeting transcripts, meeting minutes, and meeting topics. We provide quick and straightforward translation in five dif- ferent languages: French(fr), German(de), Rus- sian(ru), Italian(it), and Hindi(hi), allowing busi- nesses to save time and enhance productivity. To further optimize the translation process, we provide isometric translation [1], which generates outputs of length similar to the source length. We believe that isometric translation is the next leap towards a more synchronous auto dubbing process, which can enhance the meeting experience of non-English speaking users. Lastly, the enormous increase in online meetings and conversations has led to a massive stack of un- ordered data. It can be cumbersome for users to select the appropriate …

Content truncated to 500 words