Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks
30535Authors
nan
Abstract
AbstractNeural networks can achieve impressive performance on many natural language processing applications, but they typically need large labeled data for training and are not easily interpretable. On the other hand, symbolic rules such as regular expressions are interpretable, require no training, and often achieve decent accuracy; but rules cannot benefit from labeled data when available and hence underperform neural networks in rich-resource scenarios. In this paper, we propose a type of recurrent neural networks called FA-RNNs that combine the advantages of neural networks and regular expression rules. An FA-RNN can be converted from regular expressions and deployed in zero-shot and cold-start scenarios. It can also utilize labeled data for training to achieve improved prediction accuracy. After training, an FA-RNN often remains interpretable and can be converted back into regular expressions. We apply FA-RNNs to text classification and observe that FA-RNNs significantly outperform previous neural approaches in both zero-shot and low-resource settings and remain very competitive in rich-resource settings.
Keywords
Full Content
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 349–356, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 349 Conversational Document Prediction to Assist Customer Care Agents Jatin Ganhotra,1 Haggai Roitman,2∗Doron Cohen,1 Nathaniel Mills,1 Chulaka Gunasekara,1 Yosi Mass,1 Sachindra Joshi,1 Luis Lastras,1 David Konopnicki1 1IBM Research, 2eBay Research {jatinganhotra, wnm3, lastrasl}@us.ibm.com, hroitman@ebay.com, chulaka.gunasekara@ibm.com, {doronc, yosimass, davidko}@il.ibm.com, jsachind@in.ibm.com Abstract A frequent pattern in customer care conversa- tions is the agents responding with appropri- ate webpage URLs that address users’ needs. We study the task of predicting the documents that customer care agents can use to facilitate users’ needs. We also introduce a new pub- lic dataset1 which supports the aforementioned problem. Using this dataset and two others, we investigate state-of-the-art deep learning (DL) and information retrieval (IR) models for the task. We also analyze the practicality of such systems in terms of inference time complex- ity. Our results show that an hybrid IR+DL approach provides the best of both worlds. 1 Introduction Customer care (CC) agents play a crucial role as an organization’s main representatives to the public. Our work is motivated by the observation that, in many conversations between CC agents and users, the former tend to provide links to documents that may help resolve user issues. This is a prevalent pattern that is found in around 5-9% of all cus- tomer care conversations in multiple domains that we have reviewed. To identify such documents, the agents manually extract the keywords from the conversation and search over their customer service knowledge base (Habibi and Popescu-Belis, 2015; Ferreira et al., 2019). Table 1 shows a conversation where the agent provides a URL2 to the user. Although responding with URLs is a common pattern, automating this process to aid the agents remains underexplored in the literature. This task of Conversational Document Prediction (CDP) can be viewed as a conversational search prob- lem, where the entire conversation context or a ∗ Work done when author was at IBM Research 1The Twitter dataset is available at: https://github.com/IBM/ twitter-customer-care-document-prediction 2The terms URL and document are used interchangeably. Dialogue U: My virtual keyboard seems to float in the screen. Not sure how to undo what I just did. Can you help me please? A: We’re happy to help. To start, let us know which device you’re working with, and the OS version installed on it. U: It is an iPad A: Ok, to check version, tap Settings; General ; About. U: It’s iPad 4, 11 inch - model A1934 A: Thank you. This article can help with how to merge a split keyboard and move the keyboard for an iPad: https://support.apple.com/ en-us/HT207521. Let me know if this helps Table 1: Sample dialog from Twitter where the Agent (A) utterance includes a URL to the User (U) query. subset of it could be used as the query for retriev- ing matching documents. Compared to ad-hoc re- trieval settings, using a conversational interface, the agent/system can ask clarification questions and interactively modify the search …
Content truncated to 500 words
Labeled Text
AbstractNeural networks can achieve impressive performance on many natural language processing applications, but they typically need large labeled data for training and are not easily interpretable. On the other hand, symbolic rules such as regular expressions are interpretable, require no training, and often achieve decent accuracy; but rules cannot benefit from labeled data when available and hence underperform neural networks in rich-resource scenarios. In this paper, we propose a type of recurrent neural networks called FA-RNNs that combine the advantages of neural networks and regular expression rules. An FA-RNN can be converted from regular expressions and deployed in zero-shot and cold-start scenarios. It can also utilize labeled data for training to achieve improved prediction accuracy. After training, an FA-RNN often remains interpretable and can be converted back into regular expressions. We apply FA-RNNs to text classification and observe that FA-RNNs significantly outperform previous neural approaches in both zero-shot and low-resource settings and remain very competitive in rich-resource settings.
Publication Details
- Paper ID:
- 30535
- Year:
- 2020
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan