Обучение с подкреплением
Related Papers
Learning Optimal Dialogue Management Rules by Using Reinforcement Learning and Inductive Logic Programming
Developing dialogue systems is a complex pro- cess. In particular, designing efficient dialogue management strategies is often difficult as there are no precise guidelines to develop them and no sure …...
A Hybrid Approach to Natural Language Web Search
We describe a hybrid approach to improv- ing search performance by providing a natural language front end to a traditional keyword-based search engine. The key component of the system is …...
Comparing the Utility of State Features in Spoken Dialogue Using Reinforcement Learning
Recent work in designing spoken dialogue systems has focused on using Reinforce- ment Learning to automatically learn the best action for a system to take at any point in the …...
Learning More Effective Dialogue Strategies Using Limited Dialogue Move Features
We explore the use of restricted dialogue contexts in reinforcement learning (RL) of effective dialogue strategies for infor- mation seeking spoken dialogue systems (e.g. COMMUNICATOR (Walker et al., 2001)). The …...
Using Reinforcement Learning to Build a Better Model of Dialogue State
Given the growing complexity of tasks that spoken dialogue systems are trying to handle, Reinforcement Learning (RL) has been increasingly used as a way of au- tomatically learning the best …...
Combining Reinformation Learning with Information-State Update Rules
Reinforcement learning gives a way to learn under what circumstances to per- form which actions. However, this ap- proach lacks a formal framework for spec- ifying hand-crafted restrictions, for speci- …...
Estimating the Reliability of MDP Policies: a Confidence Interval Approach
Past approaches for using reinforcement learning to derive dialog control policies have assumed that there was enough col- lected data to derive a reliable policy. In this paper we present …...
Learning Effective Multimodal Dialogue Strategies from Wizard-of-Oz Data: Bootstrapping and Evaluation
We address two problems in the field of au- tomatic optimization of dialogue strategies: learning effective dialogue strategies when no initial data or system exists, and evaluating the result with …...
Learning to Adapt to Unknown Users: Referring Expression Generation in Spoken Dialogue Systems
We present a data-driven approach to learn user-adaptive referring expression gener- ation (REG) policies for spoken dialogue systems. Referring expressions can be dif- ficult to understand in technical domains where …...
Learning to Follow Navigational Directions
We present a system that learns to fol- low navigational natural language direc- tions. Where traditional models learn from linguistic annotation or word distri- butions, our approach is grounded in …...
Besting the Quiz Master: Crowdsourcing Incremental Classification Games
Cost-sensitive classification, where the features used in machine learning tasks have a cost, has been explored as a means of balancing knowl- edge against the expense of incrementally ob- taining …...
Evaluation of Online Dialogue Policy Learning Techniques
AbstractThe number of applied Dialogue Systems is ever increasing in several service providing and other applications as a way to efficiently and inexpensively serve large numbers of customers. A DS …...
Optimisation d’un tuteur intelligent à partir d’un jeu de données fixé (Optimization of a tutoring system from a fixed set of data) [in French]
Optimization of a tutoring system from a fixed set of data In this paper, we present a general method for optimizing a tutoring system with a target application in the …...
Comparing Multi-label Classification with Reinforcement Learning for Summarisation of Time-series Data
We present a novel approach for automatic report generation from time-series data, in the context of student feedback genera- tion. Our proposed methodology treats content selection as a multi-label (ML) …...
DINASTI: Dialogues with a Negotiating Appointment Setting Interface
AbstractThis paper describes the DINASTI (DIalogues with a Negotiating Appointment SeTting Interface) corpus, which is composed of 1734 dialogues with the French spoken dialogue system NASTIA (Negotiating Appointment SeTting InterfAce). …...
Don’t Until the Final Verb Wait: Reinforcement Learning for Simultaneous Machine Translation
We introduce a reinforcement learning- based approach to simultaneous ma- chine translation—producing a trans- lation while receiving input words— between languages with drastically dif- ferent word orders: from verb-final lan- …...
Fear the REAPER: A System for Automatic Multi-Document Summarization with Reinforcement Learning
This paper explores alternate algorithms, reward functions and feature sets for per- forming multi-document summarization using reinforcement learning with a high focus on reproducibility. We show that ROUGE results can …...
Policy Learning for Domain Selection in an Extensible Multi-domain Spoken Dialogue System
This paper proposes a Markov Decision Process and reinforcement learning based approach for domain selection in a multi- domain Spoken Dialogue System built on a distributed architecture. In the proposed …...
Deep Reinforcement Learning for Mention-Ranking Coreference Models
Coreference resolution systems are typically trained with heuristic loss functions that re- quire careful tuning. In this paper we in- stead apply reinforcement learning to directly optimize a neural mention-ranking …...
Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning
Most successful information extraction sys- tems operate with access to a large collec- tion of documents. In this work, we explore the task of acquiring and incorporating exter- nal evidence …...
Learning to Compose Neural Networks for Question Answering
We describe a question answering model that applies to both images and structured knowl- edge bases. The model uses natural lan- guage strings to automatically assemble neu- ral networks from …...
Modeling Human Reading with Neural Attention
When humans read text, they fixate some words and skip others. However, there have been few attempts to explain skipping behav- ior with computational models, as most ex- isting work …...
Agent-Aware Dropout DQN for Safe and Efficient On-line Dialogue Policy Learning
AbstractHand-crafted rules and reinforcement learning (RL) are two popular choices to obtain dialogue policy. The rule-based policy is often reliable within predefined scope but not self-adaptable, whereas RL is evolvable …...
Bootstrapping incremental dialogue systems from minimal data: the generalisation power of dialogue grammars
AbstractWe investigate an end-to-end method for automatically inducing task-based dialogue systems from small amounts of unannotated dialogue data. It combines an incremental semantic grammar - Dynamic Syntax and Type Theory …...
Coarse-to-Fine Question Answering for Long Documents
AbstractWe present a framework for question answering that can efficiently scale to longer documents while maintaining or even improving performance of state-of-the-art models. While most successful approaches for reading comprehension …...
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning
AbstractWe study the problem of learning to reason in large scale knowledge graphs (KGs). More specifically, we describe a novel reinforcement learning framework for learning multi-hop relational paths: we use …...
Integrated Learning of Dialog Strategies and Semantic Parsing
AbstractNatural language understanding and dialog management are two integral components of interactive dialog systems. Previous research has used machine learning techniques to individually optimize these components, with different forms of …...
Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication
AbstractTo enable human-robot communication and collaboration, previous works represent grounded verb semantics as the potential change of state to the physical world caused by these verbs. Grounded verb semantics are …...
Learning what to read: Focused machine reading
AbstractRecent efforts in bioinformatics have achieved tremendous progress in the machine reading of biomedical literature, and the assembly of the extracted biochemical interactions into large-scale models such as protein signaling …...
Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
AbstractWe propose to directly map raw visual observations and text input to actions for instruction execution. While existing approaches assume access to structured environment representations or use a pipeline of …...
MUSE: Modularizing Unsupervised Sense Embeddings
AbstractThis paper proposes to address the word sense ambiguity issue in an unsupervised manner, where word sense representations are learned along a word sense selection mechanism given contexts. Prior work …...
Semi-Supervised QA with Generative Domain-Adaptive Nets
AbstractWe study the problem of semi-supervised question answering—utilizing unlabeled text to boost the performance of question answering models. We propose a novel training framework, the Generative Domain-Adaptive Nets. In this …...
Should Neural Network Architecture Reflect Linguistic Structure?
AbstractI explore the hypothesis that conventional neural network models (e.g., recurrent neural networks) are incorrectly biased for making linguistically sensible generalizations when learning, and that a better class of models …...
Tackling Error Propagation through Reinforcement Learning: A Case of Greedy Dependency Parsing
AbstractError propagation is a common problem in NLP. Reinforcement learning explores erroneous states during training and can therefore be more robust when mistakes are made early in a process. In …...
Task-Oriented Query Reformulation with Reinforcement Learning
AbstractSearch engines play an important role in our everyday lives by assisting us in finding the information we need. When we input a complex query, however, results are often far …...
Towards the Use of Deep Reinforcement Learning with Global Policy for Query-based Extractive Summarisation
Supervised approaches for text summari- sation suffer from the problem of mis- match between the target labels/scores of individual sentences and the evaluation score of the final summary. Reinforcement learning …...
A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators
AbstractVisual Question Generation (VQG) aims to ask natural questions about an image automatically. Existing research focus on training model to fit the annotated data set that makes it indifferent from …...
Classical Structured Prediction Losses for Sequence to Sequence Learning
AbstractThere has been much recent work on training neural attention models at the sequence-level using either reinforcement learning-style methods or by optimizing the beam. In this paper, we survey a …...
Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems
AbstractIn this work, we present a hybrid learning method for training task-oriented dialogue systems through online user interactions. Popular methods for learning task-oriented dialogues include applying reinforcement learning with user …...
Discourse-Aware Neural Rewards for Coherent Text Generation
AbstractIn this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text. In particular, we propose to learn neural rewards …...
Learning Sentence Representations over Tree Structures for Target-Dependent Classification
AbstractTarget-dependent classification tasks, such as aspect-level sentiment analysis, perform fine-grained classifications towards specific targets. Semantic compositions over tree structures are promising for such tasks, as they can potentially capture long-distance …...
Learning with Latent Language
AbstractThe named concepts and compositional operators present in natural language provide a rich source of information about the abstractions humans use to navigate the world. Can this linguistic background knowledge …...
Neural Math Word Problem Solver with Reinforcement Learning
AbstractSequence-to-sequence model has been applied to solve math word problems. The model takes math problem descriptions as input and generates equations as output. The advantage of sequence-to-sequence model requires no …...
Representation Learning for Grounded Spatial Reasoning
AbstractThe interpretation of spatial references is highly contextual, requiring joint inference over both language and the environment. We consider the task of spatial reasoning in a simulated environment, where an …...
Source Critical Reinforcement Learning for Transferring Spoken Language Understanding to a New Language
AbstractTo deploy a spoken language understanding (SLU) model to a new language, language transferring is desired to avoid the trouble of acquiring and labeling a new big SLU corpus. An …...
An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generation
AbstractGenerating paraphrases from given sentences involves decoding words step by step from a large vocabulary. To learn a decoder, supervised learning which maximizes the likelihood of tokens always suffers from …...
An Imitation Learning Approach to Unsupervised Parsing
AbstractRecently, there has been an increasing interest in unsupervised parsers that optimize semantically oriented objectives, typically using reinforcement learning. Unfortunately, the learned trees often do not match actual syntax trees …...
Answer-guided and Semantic Coherent Question Generation in Open-domain Conversation
AbstractGenerating intriguing question is a key step towards building human-like open-domain chatbots. Although some recent works have focused on this task, compared with questions raised by humans, significant gaps remain …...
Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification
AbstractSentence simplification is the task of rewriting texts so they are easier to understand. Recent research has applied sequence-to-sequence (Seq2Seq) models to this task, focusing largely on training-time improvements via …...
Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses
AbstractSentence position is a strong feature for news summarization, since the lead often (but not always) summarizes the key points of the article. In this paper, we show that recent …...
Early Rumour Detection
AbstractRumours can spread quickly through social media, and malicious ones can bring about significant economical and social impact. Motivated by this, our paper focuses on the task of rumour detection; …...
End-to-end Deep Reinforcement Learning Based Coreference Resolution
AbstractRecent neural network models have significantly advanced the task of coreference resolution. However, current neural coreference models are usually trained with heuristic loss functions that are computed over a sequence …...
Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data
AbstractWe consider a novel question answering (QA) task where the machine needs to read from large streaming data (long documents or videos) without knowing when the questions will be given, …...
Inferential Machine Comprehension: Answering Questions by Recursively Deducing the Evidence Chain from Text
AbstractThis paper focuses on the topic of inferential machine comprehension, which aims to fully understand the meanings of given text to answer generic questions, especially the ones needed reasoning skills. …...
Learning Interpretable Negation Rules via Weak Supervision at Document Level: A Reinforcement Learning Approach
AbstractNegation scope detection is widely performed as a supervised learning task which relies upon negation labels at word level. This suffers from two key drawbacks: (1) such granular annotations are …...
Learning to Ask for Conversational Machine Learning
AbstractNatural language has recently been explored as a new medium of supervision for training machine learning models. Here, we explore learning classification tasks using language in a conversational setting – …...
Look Harder: A Neural Machine Translation Model with Hard Attention
AbstractSoft-attention based Neural Machine Translation (NMT) models have achieved promising results on several translation tasks. These models attend all the words in the source sequence for each target token, which …...
Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards
AbstractGenerating keyphrases that summarize the main points of a document is a fundamental task in natural language processing. Although existing generative models are capable of predicting multiple keyphrases for an …...
Neural Topic Model with Reinforcement Learning
AbstractIn recent years, advances in neural variational inference have achieved many successes in text processing. Examples include neural topic models which are typically built upon variational autoencoder (VAE) with an …...
Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning
AbstractText-based adventure games provide a platform on which to explore reinforcement learning in the context of a combinatorial action space, such as natural language. We present a deep reinforcement learning …...
Reinforced Training Data Selection for Domain Adaptation
AbstractSupervised models suffer from the problem of domain shifting where distribution mismatch in the data across domains greatly affect model performance. To solve the problem, training data selection (TDS) has …...
Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus
AbstractText style transfer rephrases a text from a source style (e.g., informal) to a target style (e.g., formal) while keeping its original meaning. Despite the success existing works have achieved …...
Sentence Mover’s Similarity: Automatic Evaluation for Multi-Sentence Texts
AbstractFor evaluating machine-generated texts, automatic methods hold the promise of avoiding collection of human judgments, which can be expensive and time-consuming. The most common automatic metrics, like BLEU and ROUGE, …...
Unsupervised Neural Machine Translation with Future Rewarding
AbstractIn this paper, we alleviate the local optimality of back-translation by learning a policy (takes the form of an encoder-decoder and is defined by its parameters) with future rewarding under …...
What Should I Ask? Using Conversationally Informative Rewards for Goal-oriented Visual Dialog.
AbstractThe ability to engage in goal-oriented conversations has allowed humans to gain knowledge, reduce uncertainty, and perform tasks more efficiently. Artificial agents, however, are still far behind humans in having …...
A Reinforced Generation of Adversarial Examples for Neural Machine Translation
AbstractNeural machine translation systems tend to fail on less decent inputs despite its significant efficacy, which may significantly harm the credibility of these systems—fathoming how and when neural-based systems fail …...
Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training
AbstractThe challenge of both achieving task completion by querying the knowledge base and generating human-like responses for task-oriented dialogue systems is attracting increasing research attention. In this paper, we propose …...
Answer-driven Deep Question Generation based on Reinforcement Learning
AbstractDeep question generation (DQG) aims to generate complex questions through reasoning over multiple documents. The task is challenging and underexplored. Existing methods mainly focus on enhancing document representations, with little …...
Ask to Learn: A Study on Curiosity-driven Question Generation
AbstractWe propose a novel text generation task, namely Curiosity-driven Question Generation. We start from the observation that the Question Generation task has traditionally been considered as the dual problem of …...
Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games
AbstractWe show that Reinforcement Learning (RL) methods for solving Text-Based Games (TBGs) often fail to generalize on unseen games, especially in small data regimes. To address this issue, we propose …...
Combining Cognitive Modeling and Reinforcement Learning for Clarification in Dialogue
AbstractIn many domains, dialogue systems need to work collaboratively with users to successfully reconstruct the meaning the user had in mind. In this paper, we show how cognitive models of …...
Guided Dialogue Policy Learning without Adversarial Learning in the Loop
AbstractReinforcement learning methods have emerged as a popular choice for training an efficient and effective dialogue policy. However, these methods suffer from sparse and unstable reward signals returned by a …...
Incorporating Stylistic Lexical Preferences in Generative Language Models
AbstractWhile recent advances in language modeling has resulted in powerful generation models, their generation style remains implicitly dependent on the training data and can not emulate a specific target style. …...
Knowledge-enriched, Type-constrained and Grammar-guided Question Generation over Knowledge Bases
AbstractQuestion generation over knowledge bases (KBQG) aims at generating natural-language questions about a subgraph, i.e. a set of triples. Two main challenges still face the current crop of encoder-decoder-based methods, …...
Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning
AbstractOpen attribute value extraction for emerging entities is an important but challenging task. A lot of previous works formulate the problem as a question-answering (QA) task. While the collections of …...
Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning
AbstractWalk-based models have shown their advantages in knowledge graph (KG) reasoning by achieving decent performance while providing interpretable decisions. However, the sparse reward signals offered by the KG during a …...
Learning Efficient Dialogue Policy from Demonstrations through Shaping
AbstractTraining a task-oriented dialogue agent with reinforcement learning is prohibitively expensive since it requires a large volume of interactions with users. Human demonstrations can be used to accelerate learning progress. …...
Learning Goal-oriented Dialogue Policy with opposite Agent Awareness
AbstractMost existing approaches for goal-oriented dialogue policy learning used reinforcement learning, which focuses on the target agent policy and simply treats the opposite agent policy as part of the environment. …...
Learning to Generalize for Sequential Decision Making
AbstractWe consider problems of making sequences of decisions to accomplish tasks, interacting via the medium of language. These problems are often tackled with reinforcement learning approaches. We find that these …...
Meta-Reinforced Multi-Domain State Generator for Dialogue Systems
AbstractA Dialogue State Tracker (DST) is a core component of a modular task-oriented dialogue system. Tremendous progress has been made in recent years. However, the major challenges remain. The state-of-the-art …...
Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decomposition
AbstractMany studies have applied reinforcement learning to train a dialog policy and show great promise these years. One common approach is to employ a user simulator to obtain a large …...
Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning
AbstractWhile neural sequence learning methods have made significant progress in single-document summarization (SDS), they produce unsatisfactory results on multi-document summarization (MDS). We observe two major challenges when adapting SDS advances …...
Multi-Reward based Reinforcement Learning for Neural Machine Translation
AbstractReinforcement learning (RL) has made remarkable progress in neural machine translation (NMT). However, it exists the problems with uneven sampling distribution, sparse rewards and high variance in training phase. Therefore, …...
Named Entity Recognition Only from Word Embeddings
AbstractDeep neural network models have helped named entity recognition achieve amazing performance without handcrafting features. However, existing systems require large amounts of human annotated training data. Efforts have been made …...
Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation
AbstractWe propose a method to automatically generate a domain- and task-adaptive maskings of the given text for self-supervised pre-training, such that we can effectively adapt the language model to a …...
Recognition Method of Important Words in Korean Text based on Reinforcement Learning
AbstractThe manual labeling work for constructing the Korean corpus is too time-consuming and laborious. It is difficult for low-minority languages to integrate resources. As a result, the research progress of …...
Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
AbstractIn this paper, we study machine reading comprehension (MRC) on long texts: where a model takes as inputs a lengthy document and a query, extracts a text span from the …...
Reinforced Multi-task Approach for Multi-hop Question Generation
AbstractQuestion generation (QG) attempts to solve the inverse of question answering (QA) problem by generating a natural language question given a document and an answer. While sequence to sequence neural …...
Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems
AbstractDialogue policy learning for task-oriented dialogue systems has enjoyed great progress recently mostly through employing reinforcement learning methods. However, these approaches have become very sophisticated. It is time to re-evaluate …...
RMM: A Recursive Mental Model for Dialogue Navigation
AbstractLanguage-guided robots must be able to both ask humans questions and understand answers. Much existing work focuses only on the latter. In this paper, we go beyond instruction following and …...
Slot-consistent NLG for Task-oriented Dialogue Systems with Iterative Rectification Network
AbstractData-driven approaches using neural networks have achieved promising performances in natural language generation (NLG). However, neural generators are prone to make mistakes, e.g., neglecting an input slot value and generating …...
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization
AbstractWe study unsupervised multi-document summarization evaluation metrics, which require neither human-written reference summaries nor human annotations (e.g. preferences, ratings, etc.). We propose SUPERT, which rates the quality of a summary …...
Syntactic and Semantic-driven Learning for Open Information Extraction
AbstractOne of the biggest bottlenecks in building accurate, high coverage neural open IE systems is the need for large labelled corpora. The diversity of open domain corpora and the variety …...
Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision
AbstractEvaluating the trustworthiness of a model’s prediction is essential for differentiating between ‘right for the right reasons’ and ‘right for the wrong reasons’. Identifying textual spans that determine the target …...
Zero-shot Text Classification via Reinforced Self-training
AbstractZero-shot learning has been a tough problem since no labeled data is available for unseen classes during training, especially for classes with low similarity. In this situation, transferring from seen …...
Automated Concatenation of Embeddings for Structured Prediction
AbstractPretrained contextualized embeddings are powerful word representations for structured prediction tasks. Recent work found that better word representations can be obtained by concatenating different types of embeddings. However, the selection …...
Automatic Document Sketching: Generating Drafts from Analogous Texts
The advent of large pre-trained language mod- els has made it possible to make high-quality predictions on how to add or change a sentence in a document. However, the high …...
Automatically Exposing Problems with Neural Dialog Models
AbstractNeural dialog models are known to suffer from problems such as generating unsafe and inconsistent responses. Even though these problems are crucial and prevalent, they are mostly manually identified by …...
Contrastive Multi-document Question Generation
AbstractMulti-document question generation focuses on generating a question that covers the common aspect of multiple documents. Such a model is useful in generating clarifying options. However, a naive model trained …...
Efficient Mind-Map Generation via Sequence-to-Graph and Reinforced Graph Refinement
AbstractA mind-map is a diagram that represents the central concept and key ideas in a hierarchical way. Converting plain text into a mind-map will reveal its key semantic structure and …...
Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation
AbstractThis paper addresses the problem of simultaneous machine translation (SiMT) by exploring two main concepts: (a) adaptive policies to learn a good trade-off between high translation quality and low latency; …...
How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds
AbstractWe seek to create agents that both act and communicate with other agents in pursuit of a goal. Towards this end, we extend LIGHT (Urbanek et al. 2019)—a large-scale crowd-sourced …...
Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management
AbstractFor task-oriented dialog systems, training a Reinforcement Learning (RL) based Dialog Management module suffers from low sample efficiency and slow convergence speed due to the sparse rewards in RL. To …...
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation
AbstractNeural image-to-text radiology report generation systems offer the potential to improve radiology reporting by reducing the repetitive process of report drafting and identifying possible medical errors. However, existing report generation …...
Improving Factual Consistency Between a Response and Persona Facts
AbstractNeural models for response generation produce responses that are semantically plausible but not necessarily factually consistent with facts describing the speaker’s persona. These models are trained with fully supervised learning …...
Incorporating translation quality estimation into Chinese-Korean neural machine translation
AbstractExposure bias and poor translation diversity are two common problems in neural machine trans-lation (NMT) which are caused by the general of the teacher forcing strategy for training inthe NMT …...
Low-Resource Machine Translation based on Asynchronous Dynamic Programming
AbstractReinforcement learning has been proved to be effective in handling low resource machine trans-lation tasks and different sampling methods of reinforcement learning affect the performance ofthe model. The reward for …...
Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach
AbstractThe element of repetition in cyberbullying behavior has directed recent computational studies toward detecting cyberbullying based on a social media session. In contrast to a single text, a session may …...
Multi-granularity Textual Adversarial Attack with Behavior Cloning
AbstractRecently, the textual adversarial attack models become increasingly popular due to their successful in estimating the robustness of NLP models. However, existing works have obvious deficiencies. (1)They usually consider only …...
Multiple Tasks Integration: Tagging, Syntactic and Semantic Parsing as a Single Task
AbstractDeparting from both sequential pipelines and monotask systems, we propose Multiple Tasks Integration (MTI), a multitask paradigm orthogonal to weight sharing. The essence of MTI is to process the input …...
Multitasking Inhibits Semantic Drift
AbstractWhen intelligent agents communicate to accomplish shared goals, how do these goals shape the agents’ language? We study the dynamics of learning in latent language policies (LLPs), in which instructor …...
Phrase-Level Action Reinforcement Learning for Neural Dialog Response Generation
Defining a sophisticated action space for a dia- log agent is essential for efficient training with reinforcement learning (RL). Recent work in- troduces discrete latent variables to use as an …...
Quantitative Day Trading from Natural Language using Reinforcement Learning
AbstractIt is challenging to design profitable and practical trading strategies, as stock price movements are highly stochastic, and the market is heavily influenced by chaotic data across sources like news …...
Reinforced Counterfactual Data Augmentation for Dual Sentiment Classification
AbstractData augmentation and adversarial perturbation approaches have recently achieved promising results in solving the over-fitting problem in many natural language processing (NLP) tasks including sentiment classification. However, existing studies aimed …...
Revisiting the Evaluation of End-to-end Event Extraction
Event extraction (EE) aims to harvest event in- stances from plain text, where each instance is composed of a group of event arguments with specific event roles. Existing end-to-end EE …...
Rule Augmented Unsupervised Constituency Parsing
Recently, unsupervised parsing of syntactic trees has gained considerable attention. prototypical approach to such unsupervisedA parsing employs reinforcement learning and auto-encoders. However, no mechanism en- sures that the learnt model …...
Rule-Aware Reinforcement Learning for Knowledge Graph Reasoning
Multi-hop reasoning is an effective and ex- plainable approach to predicting missing facts in Knowledge Graphs (KGs). It usually adopts the Reinforcement Learning (RL) framework and searches over the KG …...
Scheduled Dialog Policy Learning: An Automatic Curriculum Learning Framework for Task-oriented Dialog System
In reinforcement learning (RL) based task- oriented dialogue systems, users act as the en- vironment and the agent learns the policy by interacting with users. However, due to the subjectivity …...
Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphs
AbstractTemporal Knowledge Graphs (TKGs) have been developed and used in many different areas. Reasoning on TKGs that predicts potential facts (events) in the future brings great challenges to existing models. …...
Sentiment Preservation in Review Translation using Curriculum-based Re-inforcement Framework
AbstractMachine Translation (MT) systems often fail to preserve different stylistic and pragmatic properties of the source text (e.g. sentiment and emotion and gender traits and etc.) to the target and …...
TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting
AbstractTemporal knowledge graph (TKG) reasoning is a crucial task that has gained increasing research interest in recent years. Most existing methods focus on reasoning at past timestamps to complete the …...
TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference
AbstractExisting pre-trained language models (PLMs) are often computationally expensive in inference, making them impractical in various resource-limited real-world applications. To address this issue, we propose a dynamic token reduction approach …...
Transferable Dialogue Systems and User Simulators
AbstractOne of the difficulties in training dialogue systems is the lack of training data. We explore the possibility of creating dialogue data through the interaction between a dialogue system and …...
Unsupervised Learning of KB Queries in Task-Oriented Dialogs
AbstractTask-oriented dialog (TOD) systems often need to formulate knowledge base (KB) queries corresponding to the user intent and use the query results to generate system responses. Existing approaches require dialog …...
A Survey of Text Games for Reinforcement Learning Informed by Natural Language
AbstractReinforcement Learning has shown success in a number of complex virtual environments. However, many challenges still exist towards solving problems with natural language as a core component. Interactive Fiction Games …...
Active Example Selection for In-Context Learning
AbstractWith a handful of demonstration examples, large-scale language models demonstrate strong capability to perform various tasks by in-context learning from these examples, without any fine-tuning. We demonstrate that in-context learning …...
Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learning
AbstractWhen a natural language generation (NLG) component is implemented in a real-world task-oriented dialogue system, it is necessary to generate not only natural utterances as learned on training data but …...
Aligning to Social Norms and Values in Interactive Narratives
AbstractWe focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based games—environments wherein an agent perceives and interacts with a world …...
Automatic Generation of Socratic Subquestions for Teaching Math Word Problems
AbstractSocratic questioning is an educational method that allows students to discover answers to complex problems by asking them a series of thoughtful questions. Generation of didactically sound questions is challenging, …...
Composing Ci with Reinforced Non-autoregressive Text Generation
AbstractComposing Ci (also widely known as Song Ci), a special type of classical Chinese poetry, requires to follow particular format once their tune patterns are given. To automatically generate a …...
Dependency Parsing with Backtracking using Deep Reinforcement Learning
AbstractGreedy algorithms for NLP such as transition-based parsing are prone to error propagation. One way to overcome this problem is to allow the algorithm to backtrack and explore an alternative …...
Dynamic Dialogue Policy for Continual Reinforcement Learning
AbstractContinual learning is one of the key components of human learning and a necessary requirement of artificial intelligence. As dialogue can potentially span infinitely many topics and tasks, a task-oriented …...
Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning
AbstractSentence compression reduces the length of text by removing non-essential content while preserving important facts and grammaticality. Unsupervised objective driven methods for sentence compression can be used to create customized …...
Evaluating and Improving Factuality in Multimodal Abstractive Summarization
AbstractCurrent metrics for evaluating factuality for abstractive document summarization have achieved high correlations with human judgment, but they do not account for the vision modality and thus are not adequate …...
Iterative Span Selection: Self-Emergence of Resolving Orders in Semantic Role Labeling
AbstractSemantic Role Labeling (SRL) is the task of labeling semantic arguments for marked semantic predicates. Semantic arguments and their predicates are related in various distinct manners, of which certain semantic …...
Learning to Selectively Learn for Weakly Supervised Paraphrase Generation with Model-based Reinforcement Learning
AbstractParaphrase generation is an important language generation task attempting to interpret user intents and systematically generate new phrases of identical meanings to the given ones. However, the effectiveness of paraphrase …...
Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insights
AbstractInvestigating cooperativity of interlocutors is central in studying pragmatics of dialogue. Models of conversation that only assume cooperative agents fail to explain the dynamics of strategic conversations. Thus, we investigate …...
Neuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference
AbstractWe introduce a neuro-symbolic natural logic framework based on reinforcement learning with introspective revision. The model samples and rewards specific reasoning paths through policy gradient, in which the introspective revision …...
Partner Personas Generation for Dialogue Response Generation
AbstractIncorporating personas information allows diverse and engaging responses in dialogue response generation. Unfortunately, prior works have primarily focused on self personas and have overlooked the value of partner personas. Moreover, …...
PEPDS: A Polite and Empathetic Persuasive Dialogue System for Charity Donation
AbstractPersuasive conversations for a social cause often require influencing other person’s attitude or intention that may fail even with compelling arguments. The use of emotions and different types of polite …...
PLM-based World Models for Text-based Games
AbstractWorld models have improved the ability of reinforcement learning agents to operate in a sample efficient manner, by being trained to predict plausible changes in the underlying environment. As the …...
Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering
AbstractKnowledge underpins reasoning. Recent research demonstrates that when relevant knowledge is provided as additional context to commonsense question answering (QA), it can substantially enhance the performance even on top of …...
Reinforced Cross-modal Alignment for Radiology Report Generation
AbstractMedical images are widely used in clinical decision-making, where writing radiology reports is a potential application that can be enhanced by automatic solutions to alleviate physicians’ workload. In general, radiology …...
Reinforcement Learning with Large Action Spaces for Neural Machine Translation
AbstractApplying Reinforcement learning (RL) following maximum likelihood estimation (MLE) pre-training is a versatile method for enhancing neural machine translation (NMT) performance. However, recent work has argued that the gains produced …...
Selective Token Generation for Few-shot Natural Language Generation
AbstractNatural language modeling with limited training data is a challenging problem, and many algorithms make use of large-scale pretrained language models (PLMs) for this due to its great generalization ability. …...
Unsupervised Reinforcement Adaptation for Class-Imbalanced Text Classification
AbstractClass imbalance naturally exists when label distributions are not aligned across source and target domains. However, existing state-of-the-art UDA models learn domain-invariant representations across domains and evaluate primarily on class-balanced …...
ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations
AbstractContext is everything, even in commonsense moral reasoning. Changing contexts can flip the moral judgment of an action; Lying to a friend is wrong in general, but may be morally …...
Combining Parameter-efficient Modules for Task-level Generalisation
AbstractA modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. In this work, we assume that each task is …...
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback
AbstractDespite the seeming success of contemporary grounded text generation systems, they often tend to generate factually inconsistent text with respect to their input. This phenomenon is emphasized in tasks like …...
Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues
AbstractReinforcement learning has been widely adopted to model dialogue managers in task-oriented dialogues. However, the user simulator provided by state-of-the-art dialogue frameworks are only rough approximations of human behaviour. The …...
Hard Sample Aware Prompt-Tuning
AbstractPrompt-tuning based few-shot learning has garnered increasing attention in recent years due to its efficiency and promising capability. To achieve the best performance for NLP tasks with just a few …...
I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons
AbstractWe propose a novel task, G4C, to study teacher-student natural language interactions in a goal-driven and grounded environment. Dungeons and Dragons (D&D), a role-playing game, provides an ideal setting to …...
Path Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning
AbstractMulti-hop reasoning, a prevalent approach for query answering, aims at inferring new facts along reasonable paths over a knowledge graph. Reinforcement learning methods can be adopted by formulating the problem …...
Reward Gaming in Conditional Text Generation
AbstractTo align conditional text generation model outputs with desired behaviors, there has been an increasing focus on training the model using reinforcement learning (RL) with reward functions learned from human …...
Self-imitation Learning for Action Generation in Text-based Games
AbstractIn this work, we study reinforcement learning (RL) in solving text-based games. We address the challenge of combinatorial action space, by proposing a confidence-based self-imitation model to generate action candidates …...
TERL: Transformer Enhanced Reinforcement Learning for Relation Extraction
Abstract“Relation Extraction (RE) task aims to discover the semantic relation that holds between two entitiesand contributes to many applications such as knowledge graph construction and completion. Reinforcement Learning (RL) has …...