Domain- and Task-Adaptation for VaccinChatNL, a Dutch COVID-19 FAQ Answering Corpus and Classification Model

Proceedings of the 29th International Conference on Computational Linguistics, pages 3539–3549 October 12–17, 2022. 3539 Domain- and Task-Adaptation for VaccinChatNL, a Dutch COVID-19 FAQ Answering Corpus and Classification Model Jeska Buhmann, Maxime De Bruyn, Ehsan Lotfi, and Walter Daelemans CLiPS Research Center University of Antwerp, Belgium {jeska.buhmann,maxime.debruyn,ehsan.lotfi,walter.daelemans}@uantwerpen.be Abstract FAQs are important resources to find informa- tion. However, especially if a FAQ concerns many question-answer pairs, it can be a difficult and time-consuming job to find the answer you are looking for. A FAQ chatbot can ease this process by automatically retrieving the relevant answer to a user’s question. We present Vac- cinChatNL, a Dutch FAQ corpus on the topic of COVID-19 vaccination. Starting with 50 question-answer pairs we built VaccinChat, a FAQ chatbot, which we used to gather more user questions that were also annotated with the appropriate or new answer classes. This iterative process of gathering user questions, annotating them, and retraining the model with the increased data set led to a corpus that now contains 12,883 user questions divided over 181 answers. We provide the first publicly avail- able Dutch FAQ answering data set of this size with large groups of semantically equivalent human-paraphrased questions. Furthermore, our study shows that before fine-tuning a classi- fier, continued pre-training of Dutch language models with task- and/or domain-specific data improves classification results. In addition, we show that large groups of semantically simi- lar questions are important for obtaining well- performing intent classification models. 1 Introduction In quickly changing contexts, like the COVID-19 pandemic, getting access to relevant and correct in- formation in a fast and easy way is of crucial impor- tance. Although websites with frequently-asked- questions (FAQ) sections provide such information, finding the representative question that matches a user’s request can be hard and time-consuming, especially when the list of question-answer (QA) pairs is long. A FAQ chatbot does this matching for the user, speeding up the process of finding an answer to the posed question: users speak or type their questions and the system retrieves the best matching answer available. Intent: faq_ask_why Waarom zou ik mij laten vaccineren? Why would I get vaccinated? Wat zijn de voordelen? What are the advantages? Waarom moet je je laten inenten? Why do you need to get vaccinated? Ik snap niet waarom ik me moet laten vaccineren. I don’t understand why I must get vaccinated. Waarom is vaccineren belangrijk? Why is vaccination important? Hoezo moet ik mij laten inenten tegen covid? Why should I get vaccinated against covid? Waarom dringen ze zo aan op het vaccin? Why are they so insistent on the vaccine? Intent: faq_ask_certificate Krijgen we een attest na vaccinatie? Do we get a certificate after vaccination? Moet ik later kunnen bevestigen dat ik een vaccinatie gehad heb? Do I have to be able to confirm later that I have had a vaccination? Kunnen instanties me vragen te bewijzen of ik gevaccineerd ben? Can authorities ask me to prove that I have been vaccinated? Table 1: Examples of paraphrases for two …

Content truncated to 500 words