Transfer Learning Methods for Domain Adaptation in Technical Logbook Datasets

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4235–4244 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4235 Transfer Learning Methods for Domain Adaptation in Technical Logbook Datasets Farhad Akhbardeh, Marcos Zampieri, Cecilia Ovesdotter Alm, Travis Desell Rochester Institute of Technology Rochester, NY, USA {fa3019, mazgla, coagla, tjdvse}@rit.edu Abstract Event identification in technical logbooks poses challenges given the limited logbook data available in specific technical domains, the large set of possible classes, and logbook entries typically being in short form and non-standard technical language. Technical logbook data typically has both a domain, the field it comes from (e.g., automotive), and an application, what it is used for (e.g., maintenance). In order to better handle the problem of data scarcity, using a variety of technical logbook datasets, this paper investigates the benefits of using transfer learning from sources within the same domain (but different applications), from within the same application (but different domains) and from all available data. Results show that performing transfer learning within a domain provides statistically significant improvements, and in all cases but one the best performance. Interestingly, transfer learning from within the application or across the global dataset degrades results in all cases but one, which benefited from adding as much data as possible. A further analysis of the dataset similarities shows that the datasets with higher similarity scores performed better in transfer learning tasks, suggesting that this can be utilized to determine the effectiveness of adding a dataset in a transfer learning task for technical logbooks. Keywords: Transfer Learning, Domain Adaptation, Technical Logbooks, Event Classification 1. Introduction Estimating downtime and performing timely main- tenance is a key step in reducing costs, increasing safety, and improving operational efficiency in var- ious branches of engineering. Predictive mainte- nance systems use machine learning to estimate when maintenance operations should occur by using many sources of information, including historical mainte- nance records in the form of event logbooks (Carvalho et al., 2019). Event logbooks often contain short text fields that de- scribe an issue or a solution taken to address a safety problem (e.g., replacing a part). They also often in- clude issue type labels manually assigned by domain experts, making it possible to train systems to classi- fying maintenance issues automatically according to similarity (McArthur et al., 2018). The issue type and descriptions include domain-specific technical lan- guage, abbreviations, and non-standard orthography, which off-the-shelf NLP models are unable to process. This has led to development of domain-specific text pre-processing pipelines for logbook entries (Del´eger et al., 2010; Akhbardeh et al., 2020b). Another import challenge when processing logbook datasets is data scarcity. Logbooks are proprietary and not widely available making it difficult to train ro- bust predictive maintenance systems that require large amounts of data, for example, when using deep neural networks. One exception is MaintNet, an open repos- itory of predictive maintenance datasets from multiple domains, such as automotive and aviation (Akhbardeh et al., 2020a). As evidenced in Section …

Content truncated to 500 words